22 Commits

Author SHA1 Message Date
d2e08dbbec skill提交
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 6m22s
2026-07-11 15:32:56 +09:00
026d4908d8 删除
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 53s
2026-07-11 12:28:38 +09:00
31232ea899 优化迁移脚本速度
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 53s
2026-07-10 13:10:00 +09:00
b38df737e1 先短暂去除限制,上传迁移脚本
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m51s
2026-07-09 18:23:29 +09:00
346156ee9b 卡只允许支付宝支付,设备只允许微信支付,钱包充值也遵循这个规则
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m53s
2026-07-03 10:26:48 +09:00
0d79130e07 入参
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m46s
2026-07-02 17:02:13 +09:00
b3fb8c7a82 资产详情新增两个字段
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
2026-07-02 16:49:50 +09:00
44fb21eb6a 修复导入重试的问题
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m53s
2026-07-02 15:13:22 +09:00
8f738ffbe8 导入的问题修复
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m51s
2026-07-02 13:04:36 +09:00
6db152bb2f chore: 安装 ponytail lazy senior dev 规则 2026-07-01 12:26:19 +09:00
fc6af43baa 修复:溢出流量应优先记到主套餐而非加油包
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m47s
主套餐和加油包同时 Depleted 时,recordOverflowToDepletedPackage
原先取 id 最大的套餐,可能取到加油包,语义不对。

改为优先查 master_usage_id IS NULL(主套餐),找不到时
再回退取任意 Depleted 套餐。
2026-07-01 12:06:19 +09:00
52bdbbae25 修复:套餐耗尽后流量详单断档问题
问题:套餐 status=Depleted 后,queryActivePackages 查不到套餐,
DeductDataUsage 直接 return CodeNoAvailablePackage,导致上游
仍有真实流量时,tb_package_usage_daily_record 不再写入,详单断档。

修复:无 Active 套餐时,找最近一条 Depleted 套餐,将溢出流量
累加到 data_usage_mb 并写入当日 daily_record,保持详单连续性。
status 不变,不重复触发停机。
2026-07-01 12:03:39 +09:00
76f657c986 1. 套餐过期前再主动同步一次流量
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m55s
2. 生效套餐逻辑错误的问题
2026-06-30 15:54:55 +09:00
a2793f7bec 需要改造的地方 2026-06-29 18:17:06 +09:00
b5b7f9a41e 迁移
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m53s
2026-06-29 12:29:31 +09:00
b930662817 从乐观锁变成悲观锁 2026-06-29 12:29:21 +09:00
ab9da7bb33 修复
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m33s
2026-06-26 13:11:39 +09:00
781e82441d 环境变量
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Failing after 39s
2026-06-26 11:59:28 +08:00
f4b6a55b27 驳回
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Failing after 7m17s
2026-06-26 12:12:15 +09:00
8cf921f11c 驳回接口
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m55s
2026-06-25 17:10:24 +09:00
c5871b59a1 修复开放接口 wallet/package-orders 支持 IMEI 下单
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m48s
2026-06-23 11:46:02 +09:00
c0b9604515 完成
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
2026-06-23 11:05:58 +09:00
84 changed files with 2584 additions and 797 deletions

30
.agents/rules/ponytail.md Normal file
View File

@@ -0,0 +1,30 @@
# Ponytail, lazy senior dev mode
You are a lazy senior developer. Lazy means efficient, not careless. The best code is the code never written.
Before writing any code, stop at the first rung that holds:
1. Does this need to be built at all? (YAGNI)
2. Does it already exist in this codebase? Reuse the helper, util, or pattern that's already here, don't re-write it.
3. Does the standard library already do this? Use it.
4. Does a native platform feature cover it? Use it.
5. Does an already-installed dependency solve it? Use it.
6. Can this be one line? Make it one line.
7. Only then: write the minimum code that works.
The ladder runs after you understand the problem, not instead of it: read the task and the code it touches, trace the real flow end to end, then climb.
Bug fix = root cause, not symptom: a report names a symptom. Grep every caller of the function you touch and fix the shared function once — one guard there is a smaller diff than one per caller, and patching only the path the ticket names leaves a sibling caller still broken.
Rules:
- No abstractions that weren't explicitly requested.
- No new dependency if it can be avoided.
- No boilerplate nobody asked for.
- Deletion over addition. Boring over clever. Fewest files possible.
- Shortest working diff wins, but only once you understand the problem. The smallest change in the wrong place isn't lazy, it's a second bug.
- Question complex requests: "Do you actually need X, or does Y cover it?"
- Pick the edge-case-correct option when two stdlib approaches are the same size, lazy means less code, not the flimsier algorithm.
- Mark intentional simplifications with a `ponytail:` comment. If the shortcut has a known ceiling (global lock, O(n²) scan, naive heuristic), the comment names the ceiling and the upgrade path.
Not lazy about: understanding the problem (read it fully and trace the real flow before picking a rung, a small diff you don't understand is just laziness dressed up as efficiency), input validation at trust boundaries, error handling that prevents data loss, security, accessibility, the calibration real hardware needs (the platform is never the spec ideal, a clock drifts, a sensor reads off), anything explicitly requested. Lazy code without its check is unfinished: non-trivial logic leaves ONE runnable check behind, the smallest thing that fails if the logic breaks (an assert-based demo/self-check or one small test file; no frameworks, no fixtures). Trivial one-liners need no test.

View File

@@ -1,10 +1,7 @@
---
name: grill-me
description: Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
description: A relentless interview to sharpen a plan or design.
disable-model-invocation: true
---
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time.
If a question can be answered by exploring the codebase, explore the codebase instead.
Run a `/grilling` session.

View File

@@ -1,47 +0,0 @@
# ADR Format
ADRs live in `docs/adr/` and use sequential numbering: `0001-slug.md`, `0002-slug.md`, etc.
Create the `docs/adr/` directory lazily — only when the first ADR is needed.
## Template
```md
# {Short title of the decision}
{1-3 sentences: what's the context, what did we decide, and why.}
```
That's it. An ADR can be a single paragraph. The value is in recording *that* a decision was made and *why* — not in filling out sections.
## Optional sections
Only include these when they add genuine value. Most ADRs won't need them.
- **Status** frontmatter (`proposed | accepted | deprecated | superseded by ADR-NNNN`) — useful when decisions are revisited
- **Considered Options** — only when the rejected alternatives are worth remembering
- **Consequences** — only when non-obvious downstream effects need to be called out
## Numbering
Scan `docs/adr/` for the highest existing number and increment by one.
## When to offer an ADR
All three of these must be true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will look at the code and wonder "why on earth did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If a decision is easy to reverse, skip it — you'll just reverse it. If it's not surprising, nobody will wonder why. If there was no real alternative, there's nothing to record beyond "we did the obvious thing."
### What qualifies
- **Architectural shape.** "We're using a monorepo." "The write model is event-sourced, the read model is projected into Postgres."
- **Integration patterns between contexts.** "Ordering and Billing communicate via domain events, not synchronous HTTP."
- **Technology choices that carry lock-in.** Database, message bus, auth provider, deployment target. Not every library — just the ones that would take a quarter to swap out.
- **Boundary and scope decisions.** "Customer data is owned by the Customer context; other contexts reference it by ID only." The explicit no-s are as valuable as the yes-s.
- **Deliberate deviations from the obvious path.** "We're using manual SQL instead of an ORM because X." Anything where a reasonable reader would assume the opposite. These stop the next engineer from "fixing" something that was deliberate.
- **Constraints not visible in the code.** "We can't use AWS because of compliance requirements." "Response times must be under 200ms because of the partner API contract."
- **Rejected alternatives when the rejection is non-obvious.** If you considered GraphQL and picked REST for subtle reasons, record it — otherwise someone will suggest GraphQL again in six months.

View File

@@ -1,60 +0,0 @@
# CONTEXT.md Format
## Structure
```md
# {Context Name}
{One or two sentence description of what this context is and why it exists.}
## Language
**Order**:
{A one or two sentence description of the term}
_Avoid_: Purchase, transaction
**Invoice**:
A request for payment sent to a customer after delivery.
_Avoid_: Bill, payment request
**Customer**:
A person or organization that places orders.
_Avoid_: Client, buyer, account
```
## Rules
- **Be opinionated.** When multiple words exist for the same concept, pick the best one and list the others under `_Avoid_`.
- **Keep definitions tight.** One or two sentences max. Define what it IS, not what it does.
- **Only include terms specific to this project's context.** General programming concepts (timeouts, error types, utility patterns) don't belong even if the project uses them extensively. Before adding a term, ask: is this a concept unique to this context, or a general programming concept? Only the former belongs.
- **Group terms under subheadings** when natural clusters emerge. If all terms belong to a single cohesive area, a flat list is fine.
## Single vs multi-context repos
**Single context (most repos):** One `CONTEXT.md` at the repo root.
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts, where they live, and how they relate to each other:
```md
# Context Map
## Contexts
- [Ordering](./src/ordering/CONTEXT.md) — receives and tracks customer orders
- [Billing](./src/billing/CONTEXT.md) — generates invoices and processes payments
- [Fulfillment](./src/fulfillment/CONTEXT.md) — manages warehouse picking and shipping
## Relationships
- **Ordering → Fulfillment**: Ordering emits `OrderPlaced` events; Fulfillment consumes them to start picking
- **Fulfillment → Billing**: Fulfillment emits `ShipmentDispatched` events; Billing consumes them to generate invoices
- **Ordering ↔ Billing**: Shared types for `CustomerId` and `Money`
```
The skill infers which structure applies:
- If `CONTEXT-MAP.md` exists, read it to find contexts
- If only a root `CONTEXT.md` exists, single context
- If neither exists, create a root `CONTEXT.md` lazily when the first term is resolved
When multiple contexts exist, infer which one the current topic relates to. If unclear, ask.

View File

@@ -1,88 +1,7 @@
---
name: grill-with-docs
description: Grilling session that challenges your plan against the existing domain model, sharpens terminology, and updates documentation (CONTEXT.md, ADRs) inline as decisions crystallise. Use when user wants to stress-test a plan against their project's language and documented decisions.
description: A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
disable-model-invocation: true
---
<what-to-do>
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time, waiting for feedback on each question before continuing.
If a question can be answered by exploring the codebase, explore the codebase instead.
</what-to-do>
<supporting-info>
## Domain awareness
During codebase exploration, also look for existing documentation:
### File structure
Most repos have a single context:
```
/
├── CONTEXT.md
├── docs/
│ └── adr/
│ ├── 0001-event-sourced-orders.md
│ └── 0002-postgres-for-write-model.md
└── src/
```
If a `CONTEXT-MAP.md` exists at the root, the repo has multiple contexts. The map points to where each one lives:
```
/
├── CONTEXT-MAP.md
├── docs/
│ └── adr/ ← system-wide decisions
├── src/
│ ├── ordering/
│ │ ├── CONTEXT.md
│ │ └── docs/adr/ ← context-specific decisions
│ └── billing/
│ ├── CONTEXT.md
│ └── docs/adr/
```
Create files lazily — only when you have something to write. If no `CONTEXT.md` exists, create one when the first term is resolved. If no `docs/adr/` exists, create it when the first ADR is needed.
## During the session
### Challenge against the glossary
When the user uses a term that conflicts with the existing language in `CONTEXT.md`, call it out immediately. "Your glossary defines 'cancellation' as X, but you seem to mean Y — which is it?"
### Sharpen fuzzy language
When the user uses vague or overloaded terms, propose a precise canonical term. "You're saying 'account' — do you mean the Customer or the User? Those are different things."
### Discuss concrete scenarios
When domain relationships are being discussed, stress-test them with specific scenarios. Invent scenarios that probe edge cases and force the user to be precise about the boundaries between concepts.
### Cross-reference with code
When the user states how something works, check whether the code agrees. If you find a contradiction, surface it: "Your code cancels entire Orders, but you just said partial cancellation is possible — which is right?"
### Update CONTEXT.md inline
When a term is resolved, update `CONTEXT.md` right there. Don't batch these up — capture them as they happen. Use the format in [CONTEXT-FORMAT.md](./CONTEXT-FORMAT.md).
`CONTEXT.md` should be totally devoid of implementation details. Do not treat `CONTEXT.md` as a spec, a scratch pad, or a repository for implementation decisions. It is a glossary and nothing else.
### Offer ADRs sparingly
Only offer to create an ADR when all three are true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will wonder "why did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If any of the three is missing, skip the ADR. Use the format in [ADR-FORMAT.md](./ADR-FORMAT.md).
</supporting-info>
Run a `/grilling` session, using the `/domain-modeling` skill.

View File

@@ -2,6 +2,7 @@
name: handoff
description: Compact the current conversation into a handoff document for another agent to pick up.
argument-hint: "What will the next session be used for?"
disable-model-invocation: true
---
Write a handoff document summarising the current conversation so a fresh agent can continue the work. Save to the temporary directory of the user's OS - not the current workspace.

View File

@@ -1,37 +0,0 @@
# Deepening
How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [LANGUAGE.md](LANGUAGE.md) — **module**, **interface**, **seam**, **adapter**.
## Dependency categories
When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam.
### 1. In-process
Pure computation, in-memory state, no I/O. Always deepenable — merge the modules and test through the new interface directly. No adapter needed.
### 2. Local-substitutable
Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface.
### 3. Remote but owned (Ports & Adapters)
Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter.
Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."*
### 4. True external (Mock)
Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter.
## Seam discipline
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection.
- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them.
## Testing strategy: replace, don't layer
- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist — delete them.
- Write new tests at the deepened module's interface. The **interface is the test surface**.
- Tests assert on observable outcomes through the interface, not internal state.
- Tests should survive internal refactors — they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface.

View File

@@ -39,7 +39,7 @@ Repo name, date, and a compact legend: solid box = module, dashed line = seam, r
## Candidate card
The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms ([LANGUAGE.md](LANGUAGE.md)) without ceremony.
The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms (from the `/codebase-design` skill) without ceremony.
Each candidate is one `<article>`:
@@ -105,7 +105,7 @@ One larger card. Candidate name, one sentence on why, anchor link to its card. T
## Tone
Plain English, concise — but the architectural nouns and verbs come straight from [LANGUAGE.md](LANGUAGE.md). Concision is not an excuse to drift.
Plain English, concise — but the architectural nouns and verbs come straight from the `/codebase-design` skill. Concision is not an excuse to drift.
**Use exactly:** module, interface, implementation, depth, deep, shallow, seam, adapter, leverage, locality.
@@ -120,4 +120,4 @@ Plain English, concise — but the architectural nouns and verbs come straight f
**Wins bullets** name the gain in glossary terms: *"locality: bugs concentrate in one module"*, *"leverage: one interface, N call sites"*, *"interface shrinks; implementation absorbs the wrappers"*. Don't write *"easier to maintain"* or *"cleaner code"* — those terms aren't in the glossary and don't earn their place.
No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in [LANGUAGE.md](LANGUAGE.md), reach for one that is before inventing a new one.
No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in the `/codebase-design` glossary, reach for one that is before inventing a new one.

View File

@@ -1,44 +0,0 @@
# Interface Design
When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout) — your first idea is unlikely to be the best.
Uses the vocabulary in [LANGUAGE.md](LANGUAGE.md) — **module**, **interface**, **seam**, **adapter**, **leverage**.
## Process
### 1. Frame the problem space
Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate:
- The constraints any new interface would need to satisfy
- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md))
- A rough illustrative code sketch to ground the constraints — not a proposal, just a way to make the constraints concrete
Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel.
### 2. Spawn sub-agents
Spawn 3+ sub-agents in parallel using the Agent tool. Each must produce a **radically different** interface for the deepened module.
Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint:
- Agent 1: "Minimize the interface — aim for 13 entry points max. Maximise leverage per entry point."
- Agent 2: "Maximise flexibility — support many use cases and extension."
- Agent 3: "Optimise for the most common caller — make the default case trivial."
- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies."
Include both [LANGUAGE.md](LANGUAGE.md) vocabulary and CONTEXT.md vocabulary in the brief so each sub-agent names things consistently with the architecture language and the project's domain language.
Each sub-agent outputs:
1. Interface (types, methods, params — plus invariants, ordering, error modes)
2. Usage example showing how callers use it
3. What the implementation hides behind the seam
4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md))
5. Trade-offs — where leverage is high, where it's thin
### 3. Present and compare
Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**.
After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated — the user wants a strong read, not a menu.

View File

@@ -1,53 +0,0 @@
# Language
Shared vocabulary for every suggestion this skill makes. Use these terms exactly — don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point.
## Terms
**Module**
Anything with an interface and an implementation. Deliberately scale-agnostic — applies equally to a function, class, package, or tier-spanning slice.
_Avoid_: unit, component, service.
**Interface**
Everything a caller must know to use the module correctly. Includes the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics.
_Avoid_: API, signature (too narrow — those refer only to the type-level surface).
**Implementation**
What's inside a module — its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise.
**Depth**
Leverage at the interface — the amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface. A module is **shallow** when the interface is nearly as complex as the implementation.
**Seam** _(from Michael Feathers)_
A place where you can alter behaviour without editing in that place. The *location* at which a module's interface lives. Choosing where to put the seam is its own design decision, distinct from what goes behind it.
_Avoid_: boundary (overloaded with DDD's bounded context).
**Adapter**
A concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside).
**Leverage**
What callers get from depth. More capability per unit of interface they have to learn. One implementation pays back across N call sites and M tests.
**Locality**
What maintainers get from depth. Change, bugs, knowledge, and verification concentrate at one place rather than spreading across callers. Fix once, fixed everywhere.
## Principles
- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts — they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface.
- **The deletion test.** Imagine deleting the module. If complexity vanishes, the module wasn't hiding anything (it was a pass-through). If complexity reappears across N callers, the module was earning its keep.
- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape.
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it.
## Relationships
- A **Module** has exactly one **Interface** (the surface it presents to callers and tests).
- **Depth** is a property of a **Module**, measured against its **Interface**.
- A **Seam** is where a **Module**'s **Interface** lives.
- An **Adapter** sits at a **Seam** and satisfies the **Interface**.
- **Depth** produces **Leverage** for callers and **Locality** for maintainers.
## Rejected framings
- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead.
- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow — interface here includes every fact a caller must know.
- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**.

View File

@@ -1,38 +1,23 @@
---
name: improve-codebase-architecture
description: Find deepening opportunities in a codebase, informed by the domain language in CONTEXT.md and the decisions in docs/adr/. Use when the user wants to improve architecture, find refactoring opportunities, consolidate tightly-coupled modules, or make a codebase more testable and AI-navigable.
description: Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.
disable-model-invocation: true
---
# Improve Codebase Architecture
Surface architectural friction and propose **deepening opportunities** — refactors that turn shallow modules into deep ones. The aim is testability and AI-navigability.
## Glossary
This command is _informed_ by the project's domain model and built on a shared design vocabulary:
Use these terms exactly in every suggestion. Consistent language is the point — don't drift into "component," "service," "API," or "boundary." Full definitions in [LANGUAGE.md](LANGUAGE.md).
- **Module** — anything with an interface and an implementation (function, class, package, slice).
- **Interface** — everything a caller must know to use the module: types, invariants, error modes, ordering, config. Not just the type signature.
- **Implementation** — the code inside.
- **Depth** — leverage at the interface: a lot of behaviour behind a small interface. **Deep** = high leverage. **Shallow** = interface nearly as complex as the implementation.
- **Seam** — where an interface lives; a place behaviour can be altered without editing in place. (Use this, not "boundary.")
- **Adapter** — a concrete thing satisfying an interface at a seam.
- **Leverage** — what callers get from depth.
- **Locality** — what maintainers get from depth: change, bugs, knowledge concentrated in one place.
Key principles (see [LANGUAGE.md](LANGUAGE.md) for the full list):
- **Deletion test**: imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep.
- **The interface is the test surface.**
- **One adapter = hypothetical seam. Two adapters = real seam.**
This skill is _informed_ by the project's domain model. The domain language gives names to good seams; ADRs record decisions the skill should not re-litigate.
- Run the `/codebase-design` skill for the architecture vocabulary (**module**, **interface**, **depth**, **seam**, **adapter**, **leverage**, **locality**) and its principles (the deletion test, "the interface is the test surface", "one adapter = hypothetical seam, two = real"). Use these terms exactly in every suggestion — don't drift into "component," "service," "API," or "boundary."
- The domain language in `CONTEXT.md` gives names to good seams; ADRs in `docs/adr/` record decisions this command should not re-litigate.
## Process
### 1. Explore
Read the project's domain glossary and any ADRs in the area you're touching first.
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
@@ -50,7 +35,7 @@ Write a self-contained HTML file to the OS temp directory so nothing lands in th
The report uses **Tailwind via CDN** for layout and styling, and **Mermaid via CDN** for diagrams where a graph/flow/sequence reliably communicates the structure. Mix Mermaid with hand-crafted CSS/SVG visuals — use Mermaid when relationships are graph-shaped (call graphs, dependencies, sequences), and hand-built divs/SVG when you want something more editorial (mass diagrams, cross-sections, collapse animations). Each candidate gets a **before/after visualisation**. Be visual.
For each candidate, the same template as before, but rendered as a card:
For each candidate, render a card with:
- **Files** — which files/modules are involved
- **Problem** — why the current architecture is causing friction
@@ -61,7 +46,7 @@ For each candidate, the same template as before, but rendered as a card:
End the report with a **Top recommendation** section: which candidate you'd tackle first and why.
**Use CONTEXT.md vocabulary for the domain, and [LANGUAGE.md](LANGUAGE.md) vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service."
**Use CONTEXT.md vocabulary for the domain, and the `/codebase-design` vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service."
**ADR conflicts**: if a candidate contradicts an existing ADR, only surface it when the friction is real enough to warrant revisiting the ADR. Mark it clearly in the card (e.g. a warning callout: _"contradicts ADR-0007 — but worth reopening because…"_). Don't list every theoretical refactor an ADR forbids.
@@ -71,11 +56,11 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o
### 3. Grilling loop
Once the user picks a candidate, drop into a grilling conversation. Walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Side effects happen inline as decisions crystallize:
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md` — same discipline as `/grill-with-docs` (see [CONTEXT-FORMAT.md](../grill-with-docs/CONTEXT-FORMAT.md)). Create the file lazily if it doesn't exist.
- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md`. Create the file lazily if it doesn't exist.
- **Sharpening a fuzzy term during the conversation?** Update `CONTEXT.md` right there.
- **User rejects the candidate with a load-bearing reason?** Offer an ADR, framed as: _"Want me to record this as an ADR so future architecture reviews don't re-suggest it?"_ Only offer when the reason would actually be needed by a future explorer to avoid re-suggesting the same thing — skip ephemeral reasons ("not worth it right now") and self-evident ones. See [ADR-FORMAT.md](../grill-with-docs/ADR-FORMAT.md).
- **Want to explore alternative interfaces for the deepened module?** See [INTERFACE-DESIGN.md](INTERFACE-DESIGN.md).
- **User rejects the candidate with a load-bearing reason?** Offer an ADR, framed as: _"Want me to record this as an ADR so future architecture reviews don't re-suggest it?"_ Only offer when the reason would actually be needed by a future explorer to avoid re-suggesting the same thing — skip ephemeral reasons ("not worth it right now") and self-evident ones.
- **Want to explore alternative interfaces for the deepened module?** Run the `/codebase-design` skill and use its design-it-twice parallel sub-agent pattern.

View File

@@ -1,6 +1,6 @@
---
name: prototype
description: Build a throwaway prototype to flesh out a design before committing to it. Routes between two branches — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route. Use when the user wants to prototype, sanity-check a data model or state machine, mock up a UI, explore design options, or says "prototype this", "let me play with it", "try a few designs".
description: Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
---
# Prototype

View File

@@ -1,6 +1,6 @@
---
name: setup-matt-pocock-skills
description: Sets up an `## Agent skills` block in AGENTS.md/CLAUDE.md and `docs/agents/` so the engineering skills know this repo's issue tracker (GitHub or local markdown), triage label vocabulary, and domain doc layout. Run before first use of `to-issues`, `to-prd`, `triage`, `diagnose`, `tdd`, `improve-codebase-architecture`, or `zoom-out` — or if those skills appear to be missing context about the issue tracker, triage labels, or domain docs.
description: Configure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.
disable-model-invocation: true
---
@@ -44,6 +44,12 @@ Default posture: these skills were designed for GitHub. If a `git remote` points
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
If — and only if — the user picked **GitHub** or **GitLab**, ask one follow-up:
> Explainer: Open-source repos often receive feature requests as pull requests, not just issues — a PR is an issue with attached code. If you turn this on, `/triage` pulls *external* PRs into the same queue and runs them through the same labels and states as issues (collaborators' in-flight PRs are left alone). Leave it off if PRs aren't a request surface for you.
- **PRs as a request surface** — yes / no (default: no). Record the answer in `docs/agents/issue-tracker.md`. For local-markdown and other trackers, skip this question — there are no PRs.
**Section B — Triage label vocabulary.**
> Explainer: When the `triage` skill processes an incoming issue, it moves it through a state machine — needs evaluation, waiting on reporter, ready for an AFK agent to pick up, ready for a human, or won't fix. To do that, it needs to apply labels (or the equivalent in your issue tracker) that match strings *you've actually configured*. If your repo already uses different label names (e.g. `bug:triage` instead of `needs-triage`), map them here so the skill applies the right ones instead of creating duplicates.
@@ -60,7 +66,7 @@ Default: each role's string equals its name. Ask the user if they want to overri
**Section C — Domain docs.**
> Explainer: Some skills (`improve-codebase-architecture`, `diagnose`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
> Explainer: Some skills (`improve-codebase-architecture`, `diagnosing-bugs`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
Confirm the layout:
@@ -95,7 +101,7 @@ The block:
### Issue tracker
[one-line summary of where issues are tracked]. See `docs/agents/issue-tracker.md`.
[one-line summary of where issues are tracked, plus whether external PRs are a triage surface]. See `docs/agents/issue-tracker.md`.
### Triage labels

View File

@@ -8,7 +8,7 @@ How the engineering skills should consume this repo's domain documentation when
- **`CONTEXT-MAP.md`** at the repo root if it exists — it points at one `CONTEXT.md` per context. Read each one relevant to the topic.
- **`docs/adr/`** — read ADRs that touch the area you're about to work in. In multi-context repos, also check `src/<context>/docs/adr/` for context-scoped decisions.
If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The producer skill (`/grill-with-docs`) creates them lazily when terms or decisions actually get resolved.
If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill (reached via `/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms or decisions actually get resolved.
## File structure
@@ -42,7 +42,7 @@ Multi-context repo (presence of `CONTEXT-MAP.md` at the root):
When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.
If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/grill-with-docs`).
If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`).
## Flag ADR conflicts

View File

@@ -13,6 +13,18 @@ Issues and PRDs for this repo live as GitHub issues. Use the `gh` CLI for all op
Infer the repo from `git remote -v``gh` does this automatically when run inside a clone.
## Pull requests as a triage surface
**PRs as a request surface: no.** _(Set to `yes` if this repo treats external PRs as feature requests; `/triage` reads this flag.)_
When set to `yes`, PRs run through the same labels and states as issues, using the `gh pr` equivalents:
- **Read a PR**: `gh pr view <number> --comments` and `gh pr diff <number>` for the diff.
- **List external PRs for triage**: `gh pr list --state open --json number,title,body,labels,author,authorAssociation,comments` then keep only `authorAssociation` of `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR`, or `NONE` (drop `OWNER`/`MEMBER`/`COLLABORATOR`).
- **Comment / label / close**: `gh pr comment`, `gh pr edit --add-label`/`--remove-label`, `gh pr close`.
GitHub shares one number space across issues and PRs, so a bare `#42` may be either — resolve with `gh pr view 42` and fall back to `gh issue view 42`.
## When a skill says "publish to the issue tracker"
Create a GitHub issue.

View File

@@ -14,6 +14,18 @@ Issues and PRDs for this repo live as GitLab issues. Use the [`glab`](https://gi
Infer the repo from `git remote -v``glab` does this automatically when run inside a clone.
## Merge requests as a triage surface
**MRs as a request surface: no.** _(Set to `yes` if this repo treats external merge requests as feature requests; `/triage` reads this flag.)_
When set to `yes`, MRs run through the same labels and states as issues, using the `glab mr` equivalents:
- **Read an MR**: `glab mr view <number> --comments` and `glab mr diff <number>` for the diff.
- **List external MRs for triage**: `glab mr list -F json`, then keep only MRs whose author is not a project member/owner (a contributor's MR, not a maintainer's in-flight work).
- **Comment / label / close**: `glab mr note`, `glab mr update --label`/`--unlabel`, `glab mr close`.
Unlike GitHub, GitLab numbers issues and MRs separately, so `#42` is unambiguous once you know which surface the maintainer means.
## When a skill says "publish to the issue tracker"
Create a GitLab issue.

View File

@@ -1,6 +1,6 @@
---
name: tdd
description: Test-driven development with red-green-refactor loop. Use when user wants to build features or fix bugs using TDD, mentions "red-green-refactor", wants integration tests, or asks for test-first development.
description: Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
---
# Test-Driven Development
@@ -13,6 +13,8 @@ description: Test-driven development with red-green-refactor loop. Use when user
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
**Tautological tests** restate the implementation inside the assertion, so they pass by construction and give zero confidence. When the expected value is computed the way the code computes it — `expect(add(a, b)).toBe(a + b)`, snapshotting a figure you derived by hand the same way the code does, asserting a constant equals itself — the test can never disagree with the code: break the code wrong and the assertion breaks wrong with it. The expected value must come from an independent source of truth — a known-good literal, a worked example, the spec.
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
## Anti-Pattern: Horizontal Slices
@@ -44,14 +46,13 @@ RIGHT (vertical):
### 1. Planning
When exploring the codebase, use the project's domain glossary so that test names and interface vocabulary match the project's language, and respect ADRs in the area you're touching.
When exploring the codebase, read `CONTEXT.md` (if it exists) so that test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
Before writing any code:
- [ ] Confirm with user what interface changes are needed
- [ ] Confirm with user which behaviors to test (prioritize)
- [ ] Identify opportunities for [deep modules](deep-modules.md) (small interface, deep implementation)
- [ ] Design interfaces for [testability](interface-design.md)
- [ ] Identify opportunities for deep modules (small interface, deep implementation) — run the `/codebase-design` skill for the vocabulary and the testability checks
- [ ] List the behaviors to test (not implementation steps)
- [ ] Get user approval on the plan
@@ -104,6 +105,7 @@ After all tests pass, look for [refactor candidates](refactoring.md):
[ ] Test describes behavior, not implementation
[ ] Test uses public interface only
[ ] Test would survive internal refactor
[ ] Expected values are independent literals, not recomputed from the code
[ ] Code is minimal for this test
[ ] No speculative features added
```

View File

@@ -1,33 +0,0 @@
# Deep Modules
From "A Philosophy of Software Design":
**Deep module** = small interface + lots of implementation
```
┌─────────────────────┐
│ Small Interface │ ← Few methods, simple params
├─────────────────────┤
│ │
│ │
│ Deep Implementation│ ← Complex logic hidden
│ │
│ │
└─────────────────────┘
```
**Shallow module** = large interface + little implementation (avoid)
```
┌─────────────────────────────────┐
│ Large Interface │ ← Many methods, complex params
├─────────────────────────────────┤
│ Thin Implementation │ ← Just passes through
└─────────────────────────────────┘
```
When designing interfaces, ask:
- Can I reduce the number of methods?
- Can I simplify the parameters?
- Can I hide more complexity inside?

View File

@@ -1,31 +0,0 @@
# Interface Design for Testability
Good interfaces make testing natural:
1. **Accept dependencies, don't create them**
```typescript
// Testable
function processOrder(order, paymentGateway) {}
// Hard to test
function processOrder(order) {
const gateway = new StripeGateway();
}
```
2. **Return results, don't produce side effects**
```typescript
// Testable
function calculateDiscount(cart): Discount {}
// Hard to test
function applyDiscount(cart): void {
cart.total -= discount;
}
```
3. **Small surface area**
- Fewer methods = fewer tests needed
- Fewer params = simpler test setup

View File

@@ -59,3 +59,19 @@ test("createUser makes user retrievable", async () => {
expect(retrieved.name).toBe("Alice");
});
```
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
```typescript
// BAD: Expected value is recomputed the way the code computes it
test("calculateTotal sums line items", () => {
const items = [{ price: 10 }, { price: 5 }];
const expected = items.reduce((sum, i) => sum + i.price, 0);
expect(calculateTotal(items)).toBe(expected);
});
// GOOD: Expected value is an independent, known literal
test("calculateTotal sums line items", () => {
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
});
```

View File

@@ -16,6 +16,7 @@ Treat the current directory as a teaching workspace. The state of their learning
- `RESOURCES.md`: A list of resources which can be explored to ground your teaching in contextual knowledge, or to acquire knowledge and wisdom. Use the format in [RESOURCES-FORMAT.md](./RESOURCES-FORMAT.md).
- `./learning-records/*.md`: A directory of learning records, which capture what the user has learned. These are loosely equivalent to architectural decision records in software development - they capture non-obvious lessons and key insights that may need to be revised later, or drive future sessions. These should be used to calculate the zone of proximal development. They are titled `0001-<dash-case-name>.md`, where the number increments each time. Use the format in [LEARNING-RECORD-FORMAT.md](./LEARNING-RECORD-FORMAT.md).
- `./lessons/*.html`: A directory of lessons. A **lesson** is a single, self-contained HTML output that teaches one tightly-scoped thing tied to the mission. This is the primary unit of teaching in this workspace.
- `./assets/*`: Reusable **components** shared across lessons. See [Assets](#assets).
- `NOTES.md`: A scratchpad for you to jot down user preferences, or working notes.
## Philosophy
@@ -59,6 +60,14 @@ Each lesson should recommend a primary source for the user to read or watch. Thi
Each lesson should contain a reminder to ask followup questions to the agent. The agent is their teacher, and can assist with anything that's unclear.
## Assets
Lessons are built from reusable **components**, stored in `./assets/`: stylesheets, quiz widgets, simulators, diagram helpers — anything a second lesson could reuse.
Reuse is the default, not the exception. Before authoring a lesson, read `./assets/` and build from the components already there. When a lesson needs something new and reusable, write it as a component in `./assets/` and link to it — never inline code a future lesson would duplicate.
A shared stylesheet is the first component every workspace earns: every lesson links it, so the lessons look like one consistent course rather than a pile of one-offs. As the workspace grows, so should the component library.
## The Mission
Every lesson should be tied into the mission - the reason that the user is interested in learning about the topic.

View File

@@ -1,6 +1,7 @@
---
name: to-issues
description: Break a plan, spec, or PRD into independently-grabbable issues on the project issue tracker using tracer-bullet vertical slices. Use when user wants to convert a plan into issues, create implementation tickets, or break down work into issues.
description: Break a plan, spec, or PRD into independently-grabbable issues on the project issue tracker using tracer-bullet vertical slices.
disable-model-invocation: true
---
# To Issues
@@ -19,16 +20,18 @@ Work from whatever is already in the conversation context. If the user passes an
If you have not already explored the codebase, do so to understand the current state of the code. Issue titles and descriptions should use the project's domain glossary vocabulary, and respect ADRs in the area you're touching.
Look for opportunities to prefactor the code to make the implementation easier. "Make the change easy, then make the easy change."
### 3. Draft vertical slices
Break the plan into **tracer bullet** issues. Each issue is a thin vertical slice that cuts through ALL integration layers end-to-end, NOT a horizontal slice of one layer.
Slices may be 'HITL' or 'AFK'. HITL slices require human interaction, such as an architectural decision or a design review. AFK slices can be implemented and merged without human interaction. Prefer AFK over HITL where possible.
<vertical-slice-rules>
- Each slice delivers a narrow but COMPLETE path through every layer (schema, API, UI, tests)
- A completed slice is demoable or verifiable on its own
- Prefer many thin slices over few thick ones
- Any prefactoring should be done first
</vertical-slice-rules>
### 4. Quiz the user
@@ -36,7 +39,6 @@ Slices may be 'HITL' or 'AFK'. HITL slices require human interaction, such as an
Present the proposed breakdown as a numbered list. For each slice, show:
- **Title**: short descriptive name
- **Type**: HITL / AFK
- **Blocked by**: which other slices (if any) must complete first
- **User stories covered**: which user stories this addresses (if the source material has them)
@@ -45,7 +47,6 @@ Ask the user:
- Does the granularity feel right? (too coarse / too fine)
- Are the dependency relationships correct?
- Should any slices be merged or split further?
- Are the correct slices marked as HITL and AFK?
Iterate until the user approves the breakdown.

View File

@@ -1,6 +1,7 @@
---
name: to-prd
description: Turn the current conversation context into a PRD and publish it to the project issue tracker. Use when user wants to create a PRD from the current context.
description: Turn the current conversation into a PRD and publish it to the project issue tracker — no interview, just synthesis of what you've already discussed.
disable-model-invocation: true
---
This skill takes the current conversation context and codebase understanding and produces a PRD. Do NOT interview the user — just synthesize what you already know.
@@ -11,7 +12,7 @@ The issue tracker and triage label vocabulary should have been provided to you
1. Explore the repo to understand the current state of the codebase, if you haven't already. Use the project's domain glossary vocabulary throughout the PRD, and respect any ADRs in the area you're touching.
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can.
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can. The fewer seams across the codebase, the better - the ideal number is one.
Check with the user that these seams match their expectations.

View File

@@ -1,6 +1,8 @@
# Writing Agent Briefs
An agent brief is a structured comment posted on a GitHub issue when it moves to `ready-for-agent`. It is the authoritative specification that an AFK agent will work from. The original issue body and discussion are context — the agent brief is the contract.
An agent brief is a structured comment posted on a GitHub issue or PR when it moves to `ready-for-agent`. It is the authoritative specification that an AFK agent will work from. The original body and discussion are context — the agent brief is the contract.
The brief states **what the agent should do**, which stretches to both surfaces: for an issue, that's building the change from nothing; for a PR, it's what's left to do *to the existing diff* — finish it, close gaps, address review points. Same principles either way; the PR example below shows the difference.
## Principles
@@ -143,6 +145,43 @@ checked for matches.
- Bug reports (only enhancement rejections go to `.out-of-scope/`)
```
### Good agent brief (PR)
For a PR, "Current behavior" describes the state of the diff, and the brief asks the agent to finish or fix it rather than build from scratch.
```markdown
## Agent Brief
**Category:** enhancement
**Summary:** Finish the contributor's `--json` output flag for `triage list`
**Current behavior:**
The PR adds a `--json` flag that serializes the issue list to JSON. The happy
path works and the diff matches the project's command structure. Two gaps
remain: errors are still printed as human text (not JSON), and the new flag has
no test coverage.
**Desired behavior:**
With `--json`, all output — including errors — is well-formed JSON on stdout,
and the command's exit codes are unchanged. The existing human-readable output
is untouched when the flag is absent.
**Key interfaces:**
- The command's error path should emit `{ "error": string }` under `--json`
instead of the plain-text error
- Reuse the existing serializer the PR already added; don't introduce a second
**Acceptance criteria:**
- [ ] `triage list --json` emits valid JSON for both success and error cases
- [ ] Exit codes match the non-JSON command
- [ ] A test covers the `--json` success output and one error case
- [ ] Default (non-JSON) output is byte-for-byte unchanged
**Out of scope:**
- Adding `--json` to any other command
- Changing the JSON shape of the success payload the PR already defined
```
### Bad agent brief
```markdown

View File

@@ -83,7 +83,11 @@ The maintainer may:
## When to write to `.out-of-scope/`
Only when an **enhancement** (not a bug) is rejected as `wontfix`. The flow:
Only when an **enhancement** (not a bug) is *rejected* as `wontfix`. This applies to enhancement PRs exactly as it does to issues — a rejected PR is recorded here so the same request doesn't return as fresh code.
Do **not** write here when something is closed as `wontfix` because it's **already implemented**. That's a built feature, not a rejected one; recording it would poison the dedup checks with false rejections. Instead, the closing comment points to where the feature already lives.
The flow:
1. Maintainer decides a feature request is out of scope
2. Check if a matching `.out-of-scope/` file already exists

View File

@@ -1,12 +1,15 @@
---
name: triage
description: Triage issues through a state machine driven by triage roles. Use when user wants to create an issue, triage issues, review incoming bugs or feature requests, prepare issues for an AFK agent, or manage issue workflow.
description: Move issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.
disable-model-invocation: true
---
# Triage
Move issues on the project issue tracker through a small state machine of triage roles.
If this repo treats external pull requests as a request surface (see the issue-tracker config), triage covers them too: **a PR is an issue with attached code** — same roles, same states, same machine, with a few deltas marked "for a PR" below. Resolve a bare `#42` to an issue or PR per the tracker config.
Every comment or issue posted to the issue tracker during triage **must** start with this disclaimer:
```
@@ -33,6 +36,8 @@ Five **state** roles:
- `ready-for-human` — needs human implementation
- `wontfix` — will not be actioned
For a PR, the same states read against the attached code: `ready-for-agent` means a brief is attached and an agent should take the next step on the diff; `ready-for-human` means it's ready for a human to merge.
Every triaged issue should carry exactly one category role and one state role. If state roles conflict, flag it and ask the maintainer before doing anything else.
These are canonical role names — the actual label strings used in the issue tracker may differ. The mapping should have been provided to you - run `/setup-matt-pocock-skills` if not.
@@ -44,7 +49,7 @@ State transitions: an unlabeled issue normally goes to `needs-triage` first; fro
The maintainer invokes `/triage` and describes what they want in natural language. Interpret the request and act. Examples:
- "Show me anything that needs my attention"
- "Let's look at #42"
- "Let's look at #42" (issue or PR)
- "Move #42 to ready-for-agent"
- "What's ready for agents to pick up?"
@@ -56,24 +61,28 @@ Query the issue tracker and present three buckets, oldest first:
2. **`needs-triage`** — evaluation in progress.
3. **`needs-info` with reporter activity since the last triage notes** — needs re-evaluation.
Show counts and a one-line summary per issue. Let the maintainer pick.
When PRs are in scope, include external PRs in these buckets and tag each line `[PR]` or `[issue]`. Discovery surfaces only *external* PRs (the tracker config defines who counts as external) — a collaborator's in-flight PR is not triage work. This filter is discovery-only; an explicitly named PR is always triaged regardless of author.
## Triage a specific issue
Show counts and a one-line summary per item. Let the maintainer pick.
1. **Gather context.** Read the full issue (body, comments, labels, reporter, dates). Parse any prior triage notes so you don't re-ask resolved questions. Explore the codebase using the project's domain glossary, respecting ADRs in the area. Read `.out-of-scope/*.md` and surface any prior rejection that resembles this issue.
## Triage a specific issue or PR
2. **Recommend.** Tell the maintainer your category and state recommendation with reasoning, plus a brief codebase summary relevant to the issue. Wait for direction.
1. **Gather context.** Read the full issue or PR (body, comments, labels, author, dates; for a PR, the diff too). Parse any prior triage notes so you don't re-ask resolved questions. Explore the codebase using the project's domain glossary, respecting ADRs in the area. Run two checks against the codebase: (a) **redundancy** — search for an existing implementation of the requested behavior by domain concept (not just the request's wording), and report where you looked. If found, it's an already-implemented `wontfix` (step 5). (b) **prior rejection** — read `.out-of-scope/*.md` and surface any that resembles this request.
3. **Reproduce (bugs only).** Before any grilling, attempt reproduction: read the reporter's steps, trace the relevant code, run tests or commands. Report what happened — successful repro with code path, failed repro, or insufficient detail (a strong `needs-info` signal). A confirmed repro makes a much stronger agent brief.
2. **Recommend.** Tell the maintainer your category and state recommendation with reasoning, plus a brief codebase summary relevant to the request — including whether it's already implemented. Wait for direction.
4. **Grill (if needed).** If the issue needs fleshing out, run a `/grill-with-docs` session.
3. **Verify the claim.** Before any grilling, check that the claim holds up. For a bug, reproduce it from the reporter's steps. For a PR, confirm the diff does what it claims — check it out, run the relevant tests or commands. Report what happened: confirmed (with code path), failed, or insufficient detail (a strong `needs-info` signal). A confirmed verification makes a much stronger agent brief.
4. **Grill (if needed).** If the request needs fleshing out, run the `/grilling` and `/domain-modeling` skills together — grill it into shape one question at a time, sharpening domain terms and updating `CONTEXT.md`/ADRs inline as decisions land.
5. **Apply the outcome:**
- `ready-for-agent` — post an agent brief comment ([AGENT-BRIEF.md](AGENT-BRIEF.md)).
- `ready-for-human` — same structure as an agent brief, but note why it can't be delegated (judgment calls, external access, design decisions, manual testing).
- `needs-info` — post triage notes (template below).
- `wontfix` (bug) — polite explanation, then close.
- `wontfix` (enhancement) — write to `.out-of-scope/`, link to it from a comment, then close ([OUT-OF-SCOPE.md](OUT-OF-SCOPE.md)).
- `wontfix` — close, with the comment depending on *why*:
- **Already implemented** — the change already exists in the codebase. Point to where it lives; do **not** write to `.out-of-scope/` (that KB is for *rejected* requests, not built ones).
- **Rejected (bug)** — polite explanation, then close.
- **Rejected (enhancement)** — write to `.out-of-scope/`, link to it from a comment, then close ([OUT-OF-SCOPE.md](OUT-OF-SCOPE.md)).
- `needs-triage` — apply the role. Optional comment if there's partial progress.
## Quick state override
@@ -100,4 +109,4 @@ Capture everything resolved during grilling under "established so far" so the wo
## Resuming a previous session
If prior triage notes exist on the issue, read them, check whether the reporter has answered any outstanding questions, and present an updated picture before continuing. Don't re-ask resolved questions.
If prior triage notes exist on the issue or PR, read them, check whether the reporter has answered any outstanding questions, and present an updated picture before continuing. Don't re-ask resolved questions.

11
.env Normal file
View File

@@ -0,0 +1,11 @@
MIGRATIONS_DIR=migrations
DB_HOST=cxd.whcxd.cn
DB_PORT=16159
DB_USER=erp_pgsql
DB_PASSWORD=erp_2025
DB_NAME=junhong_cmp_test
DB_SSLMODE=disable
GOOGLE_GEMINI_BASE_URL="http://45.155.220.179:8317" # 根据实际填写你服务器的ip地址或者域名
GEMINI_API_KEY="sk-VoNbvr6aGpjvZX64rvhrwowrZrCgtGuX9oxykIy8F1DBg"
GOOGLE_GENAI_USE_GCA="true"
GEMINI_MODEL="gemini-3-pro-preview" # 如果你有gemini3权限可以填 gemini-3-pro-preview

93
.env.local Normal file
View File

@@ -0,0 +1,93 @@
# ============================================================================
# 君鸿卡管系统 - 本地开发环境变量
# 生成时间: 2026-01-30 17:45:14
# ============================================================================
# 使用方法:
# source .env.local && go run cmd/api/main.go
# 或者:
# ./scripts/run-local.sh
# ============================================================================
# ----------------------------------------------------------------------------
# 数据库配置(必填)
# ----------------------------------------------------------------------------
export JUNHONG_DATABASE_HOST="cxd.whcxd.cn"
export JUNHONG_DATABASE_PORT="16159"
export JUNHONG_DATABASE_USER="erp_pgsql"
export JUNHONG_DATABASE_PASSWORD="erp_2025"
export JUNHONG_DATABASE_DBNAME="junhong_cmp_test"
export JUNHONG_DATABASE_SSLMODE="disable"
# ----------------------------------------------------------------------------
# Redis 配置(必填)
# ----------------------------------------------------------------------------
export JUNHONG_REDIS_ADDRESS="cxd.whcxd.cn"
export JUNHONG_REDIS_PORT="16299"
export JUNHONG_REDIS_PASSWORD="cpNbWtAaqgo1YJmbMp3h"
export JUNHONG_REDIS_DB="6"
# ----------------------------------------------------------------------------
# JWT 配置(必填)
# ----------------------------------------------------------------------------
export JUNHONG_JWT_SECRET_KEY="dev-secret-key-for-testing-only-32chars!"
# ----------------------------------------------------------------------------
# 客户端配置(可选)
# ----------------------------------------------------------------------------
# 是否强制 C 端用户绑定手机号true: 强制false: 不强制)
export JUNHONG_CLIENT_REQUIRE_PHONE_BINDING="true"
# ----------------------------------------------------------------------------
# 服务器配置
# ----------------------------------------------------------------------------
export JUNHONG_SERVER_ADDRESS=":3000"
# ----------------------------------------------------------------------------
# 日志配置
# ----------------------------------------------------------------------------
export JUNHONG_LOGGING_LEVEL="debug"
export JUNHONG_LOGGING_DEVELOPMENT="true"
export JUNHONG_LOGGING_APP_LOG_FILENAME="logs/app.log"
export JUNHONG_LOGGING_ACCESS_LOG_FILENAME="logs/access.log"
# ----------------------------------------------------------------------------
# Gateway 服务配置
# ----------------------------------------------------------------------------
export JUNHONG_GATEWAY_BASE_URL="https://open.whjhft.com/openapi"
export JUNHONG_GATEWAY_APP_ID="LfjL0WjUqpwkItQ0"
export JUNHONG_GATEWAY_APP_SECRET="K0DYuWzbRE6zg5bX"
export JUNHONG_GATEWAY_TIMEOUT="30"
# ----------------------------------------------------------------------------
# 对象存储配置
# ----------------------------------------------------------------------------
export JUNHONG_STORAGE_PROVIDER="s3"
export JUNHONG_STORAGE_S3_ENDPOINT="http://obs-helf.cucloud.cn"
export JUNHONG_STORAGE_S3_REGION="cn-langfang-2"
export JUNHONG_STORAGE_S3_BUCKET="cmp"
export JUNHONG_STORAGE_S3_ACCESS_KEY_ID="598F558CF6FF46E79D1CFC607852378C9523"
export JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY="8393425DCB2F48F1914FF39DCBC6C7B17325"
export JUNHONG_STORAGE_S3_USE_SSL="false"
export JUNHONG_STORAGE_S3_PATH_STYLE="true"
# ----------------------------------------------------------------------------
# 迁移工具兼容配置(用于 scripts/migrate.sh
# ----------------------------------------------------------------------------
export MIGRATIONS_DIR="migrations"
export DB_HOST="cxd.whcxd.cn"
export DB_PORT="16159"
export DB_USER="erp_pgsql"
export DB_PASSWORD="erp_2025"
export DB_NAME="junhong_cmp_test"
export DB_SSLMODE="disable"
# ----------------------------------------------------------------------------
# 短信服务配置
# ----------------------------------------------------------------------------
export JUNHONG_SMS_GATEWAY_URL="https://gateway.sms.whjhft.com:8443"
export JUNHONG_SMS_USERNAME="JH0001"
export JUNHONG_SMS_PASSWORD="wwR8E4qnL6F0"
export JUNHONG_SMS_SIGNATURE="【JHFTIOT】"
JUNHONG_QUEUE_CONCURRENCY=1000

236
.env.prod Normal file
View File

@@ -0,0 +1,236 @@
# ==========================================================================
# 君鸿卡管系统 - 本地开发环境变量
# 生成时间: 2026-05-07 11:56:19
# 配置依据: pkg/config/defaults/config.yaml + pkg/config/loader.go
# ==========================================================================
# 使用方法:
# source .env.prod && go run cmd/api/main.go
# 或者:
# ./scripts/run-local.sh
#
# 说明:
# - 未注释的 是本地启动常用或必填配置。
# - 注释掉的 是当前有默认值、按需启用或当前主流程未使用的配置。
# - 取消注释前请先阅读上方说明,避免覆盖嵌入配置默认值。
# ==========================================================================
# ----------------------------------------------------------------------------
# 数据库配置(必填)
# ----------------------------------------------------------------------------
# 数据库主机地址;应用启动必填。
export JUNHONG_DATABASE_HOST=cxd.whcxd.cn
# 数据库端口;默认 5432。
export JUNHONG_DATABASE_PORT=16159
# 数据库用户名;应用启动必填。
export JUNHONG_DATABASE_USER=junhong_cmp
# 数据库密码;应用启动必填,敏感信息请勿提交。
export JUNHONG_DATABASE_PASSWORD=CtCom1zBzPQbpVNf3rCNxH
# 数据库名称;应用启动必填。
export JUNHONG_DATABASE_DBNAME=junhong_cmp_prod
# 数据库 SSL 模式;本地通常使用 disable。
export JUNHONG_DATABASE_SSLMODE=disable
# ----------------------------------------------------------------------------
# Redis 配置(必填)
# ----------------------------------------------------------------------------
# Redis 主机地址;应用启动必填。
export JUNHONG_REDIS_ADDRESS=192.168.1.22
# Redis 端口;默认 6379。
export JUNHONG_REDIS_PORT=6379
# Redis 密码;无密码时留空。
export JUNHONG_REDIS_PASSWORD=Bm500vXEyLg0RUzw7YcyPjbRifhz
# Redis 数据库编号;默认 0。
export JUNHONG_REDIS_DB=0
# ----------------------------------------------------------------------------
# JWT 配置(必填)
# ----------------------------------------------------------------------------
# JWT 签名密钥;必须至少 32 字符,生产环境必须单独生成。
export JUNHONG_JWT_SECRET_KEY=a0xzpGhR3zB54JAiEJtoKnjwgAGMLmRw
# C 端 JWT Token 有效期;默认 24h通常无需覆盖。
export JUNHONG_JWT_TOKEN_DURATION=24h
# B 端访问令牌 TTL默认 24h。
export JUNHONG_JWT_ACCESS_TOKEN_TTL=24h
# B 端刷新令牌 TTL默认 168h。
export JUNHONG_JWT_REFRESH_TOKEN_TTL=168h
# ----------------------------------------------------------------------------
# 服务器配置
# ----------------------------------------------------------------------------
# 服务监听地址;本地默认 :3000。
export JUNHONG_SERVER_ADDRESS=:3527
# 读取请求超时时间;默认 30s通常无需覆盖。
export JUNHONG_SERVER_READ_TIMEOUT=30s
# 写响应超时时间;默认 30s通常无需覆盖。
export JUNHONG_SERVER_WRITE_TIMEOUT=30s
# 优雅关闭超时时间;默认 30s。
export JUNHONG_SERVER_SHUTDOWN_TIMEOUT=30s
# Fiber 预分叉模式;本地和 macOS 开发环境通常不要启用。
# JUNHONG_SERVER_PREFORK=false
# ----------------------------------------------------------------------------
# 日志配置
# ----------------------------------------------------------------------------
# 日志级别;可选 debug/info/warn/error。
export JUNHONG_LOGGING_LEVEL=debug
# 本地开发模式日志;生产环境应使用 false。
export JUNHONG_LOGGING_DEVELOPMENT=true
# 应用日志文件路径;本地写入项目 logs 目录。
export JUNHONG_LOGGING_APP_LOG_FILENAME=logs/app.log
# 访问日志文件路径;本地写入项目 logs 目录。
export JUNHONG_LOGGING_ACCESS_LOG_FILENAME=logs/access.log
# 应用日志单文件最大大小MB默认 100。
export JUNHONG_LOGGING_APP_LOG_MAX_SIZE=100
# 应用日志最多保留的旧文件数;默认 3。
export JUNHONG_LOGGING_APP_LOG_MAX_BACKUPS=15
# 应用日志最多保留天数;默认 7。
export JUNHONG_LOGGING_APP_LOG_MAX_AGE=14
# 是否压缩应用日志轮转文件;默认 true。
export JUNHONG_LOGGING_APP_LOG_COMPRESS=true
# 访问日志单文件最大大小MB默认 100。
export JUNHONG_LOGGING_ACCESS_LOG_MAX_SIZE=100
# 访问日志最多保留的旧文件数;默认 3。
export JUNHONG_LOGGING_ACCESS_LOG_MAX_BACKUPS=15
# 访问日志最多保留天数;默认 7。
export JUNHONG_LOGGING_ACCESS_LOG_MAX_AGE=14
# 是否压缩访问日志轮转文件;默认 true。
export JUNHONG_LOGGING_ACCESS_LOG_COMPRESS=true
# ----------------------------------------------------------------------------
# 客户端配置
# ----------------------------------------------------------------------------
# 是否强制 C 端用户绑定手机号;当前默认 true。
export JUNHONG_CLIENT_REQUIRE_PHONE_BINDING=true
# ----------------------------------------------------------------------------
# 数据库连接池配置(可选)
# ----------------------------------------------------------------------------
# 数据库最大打开连接数;默认 25本地通常不用改。
export JUNHONG_DATABASE_MAX_OPEN_CONNS=25
# 数据库最大空闲连接数;默认 10。
export JUNHONG_DATABASE_MAX_IDLE_CONNS=10
# 数据库连接最大生命周期;默认 5m。
export JUNHONG_DATABASE_CONN_MAX_LIFETIME=5m
# ----------------------------------------------------------------------------
# Redis 连接池配置(可选)
# ----------------------------------------------------------------------------
# Redis 连接池大小;默认 10。
export JUNHONG_REDIS_POOL_SIZE=20
# Redis 最小空闲连接数;默认 5。
export JUNHONG_REDIS_MIN_IDLE_CONNS=5
# Redis 建连超时;默认 5s。
export JUNHONG_REDIS_DIAL_TIMEOUT=5s
# Redis 读取超时;默认 3s。
export JUNHONG_REDIS_READ_TIMEOUT=3s
# Redis 写入超时;默认 3s。
export JUNHONG_REDIS_WRITE_TIMEOUT=3s
# ----------------------------------------------------------------------------
# 任务队列配置(可选)
# ----------------------------------------------------------------------------
# Worker 并发数;默认 10。
# JUNHONG_QUEUE_CONCURRENCY=10
# 任务最大重试次数;默认 5。
# JUNHONG_QUEUE_RETRY_MAX=5
# 单个任务超时时间;默认 10m。
# JUNHONG_QUEUE_TIMEOUT=10m
# 队列权重 critical/default/low 当前由嵌入配置 queue.queues 管理,未绑定为环境变量。
# ----------------------------------------------------------------------------
# 限流中间件配置(可选)
# ----------------------------------------------------------------------------
# 是否启用全局限流;默认关闭。
# JUNHONG_MIDDLEWARE_ENABLE_RATE_LIMITER=false
# 限流窗口内最大请求数;默认 100。
# JUNHONG_MIDDLEWARE_RATE_LIMITER_MAX=100
# 限流时间窗口;默认 1m。
# JUNHONG_MIDDLEWARE_RATE_LIMITER_EXPIRATION=1m
# 限流存储后端;可选 memory 或 redis。
# JUNHONG_MIDDLEWARE_RATE_LIMITER_STORAGE=memory
# ----------------------------------------------------------------------------
# 轮询配置(可选)
# ----------------------------------------------------------------------------
# 是否输出轮询详细日志;默认 false排查上游数据时再开启。
export JUNHONG_POLLING_VERBOSE_LOG=true
# 是否启用 C 端实名自动触发;默认 true。
# JUNHONG_POLLING_AUTO_TRIGGER_ENABLE_AUTO_TRIGGER=true
# 自动触发使用的系统用户 ID生产建议改成专用平台账号。
export JUNHONG_POLLING_AUTO_TRIGGER_AUTO_TRIGGER_SYSTEM_USER_ID=1
# ----------------------------------------------------------------------------
# Worker 运行配置(可选)
# ----------------------------------------------------------------------------
# Worker 角色;单实例默认 all多实例可用 leader/consumer。
# JUNHONG_WORKER_ROLE=all
# Worker 实例名称;多实例部署时用于区分日志。
# JUNHONG_WORKER_INSTANCE_NAME=''
# ----------------------------------------------------------------------------
# Gateway 服务配置
# ----------------------------------------------------------------------------
# Gateway API 基础 URL已按本次输入覆盖嵌入默认值。
export JUNHONG_GATEWAY_BASE_URL=https://open.whjhft.com/openapi
# Gateway 应用 ID敏感信息请勿提交。
export JUNHONG_GATEWAY_APP_ID=LfjL0WjUqpwkItQ0
# Gateway 应用密钥;敏感信息请勿提交。
export JUNHONG_GATEWAY_APP_SECRET=K0DYuWzbRE6zg5bX
# Gateway 请求超时时间(秒);有效范围 5-300。
export JUNHONG_GATEWAY_TIMEOUT=30
# ----------------------------------------------------------------------------
# 短信服务配置(可选)
# ----------------------------------------------------------------------------
# 短信服务未配置时系统会跳过短信客户端初始化;需要短信验证码时再取消注释。
# 短信网关地址;设置后 username/password/signature 也必须填写。
export JUNHONG_SMS_GATEWAY_URL='https://gateway.sms.whjhft.com:8443'
# 短信账号用户名。
export JUNHONG_SMS_USERNAME='JH0001'
# 短信账号密码;敏感信息请勿提交。
export JUNHONG_SMS_PASSWORD='wwR8E4qnL6F0'
# 短信签名,例如【君鸿】。
export JUNHONG_SMS_SIGNATURE='【JHFTIOT】'
# 短信请求超时时间;默认 10s。
export JUNHONG_SMS_TIMEOUT=10s
# ----------------------------------------------------------------------------
# 对象存储配置(可选)
# ----------------------------------------------------------------------------
# 对象存储提供商;当前仅支持 s3 兼容服务。
export JUNHONG_STORAGE_PROVIDER=s3
# 临时文件目录;用于导入导出等临时文件。
export JUNHONG_STORAGE_TEMP_DIR=/tmp/junhong-storage
# S3 端点;配置后会初始化对象存储服务。
export JUNHONG_STORAGE_S3_ENDPOINT=https://obs-helf.cucloud.cn
# S3 区域。
export JUNHONG_STORAGE_S3_REGION=cn-langfang-2
# S3 存储桶名称。
export JUNHONG_STORAGE_S3_BUCKET=cmp
# S3 Access Key ID敏感信息请勿提交。
export JUNHONG_STORAGE_S3_ACCESS_KEY_ID=598F558CF6FF46E79D1CFC607852378C9523
# S3 Secret Access Key敏感信息请勿提交。
export JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY=8393425DCB2F48F1914FF39DCBC6C7B17325
# 是否使用 SSL本地兼容服务常用 false。
export JUNHONG_STORAGE_S3_USE_SSL=false
# 是否使用路径风格;兼容 S3 服务通常使用 true。
export JUNHONG_STORAGE_S3_PATH_STYLE=true
# 预签名上传 URL 有效期;默认 15m。
export JUNHONG_STORAGE_PRESIGN_UPLOAD_EXPIRES=15m
# 预签名下载 URL 有效期;默认 24h。
export JUNHONG_STORAGE_PRESIGN_DOWNLOAD_EXPIRES=24h
# ----------------------------------------------------------------------------
# 默认超级管理员配置(可选)
# ----------------------------------------------------------------------------
# 不配置时会使用 pkg/constants 中的默认超管账号;生产环境建议在首次启动前覆盖。
# 默认超级管理员用户名;留空则使用代码内置默认值 admin。
export JUNHONG_DEFAULT_ADMIN_USERNAME='admin'
# 默认超级管理员密码;留空则使用代码内置默认值 Admin@123456。
export JUNHONG_DEFAULT_ADMIN_PASSWORD='Admin@123456'
# 默认超级管理员手机号;留空则使用代码内置默认值 13800000000。
# JUNHONG_DEFAULT_ADMIN_PHONE=''
export JUNHONG_MIDDLEWARE_CORS_ENABLED=true
export JUNHONG_MIDDLEWARE_CORS_ALLOW_ORIGINS=https://cmp-admin.xm-iot.cn,https://cmp-agent.xm-iot.cn,https://cmp-c.xm-iot.cn
export JUNHONG_MIDDLEWARE_CORS_ALLOW_CREDENTIALS=true

3
.gitignore vendored
View File

@@ -26,8 +26,6 @@ vendor/
.cache/
# Environment variables
.env
.env.*
!.env.example
# Log files
@@ -75,7 +73,6 @@ __debug_bin1621385388
docs/admin-openapi.yaml
/api
/gendocs
.env.local
/worker
.opencode/skills/json-canvas
.opencode/skills/obsidian-bases

View File

@@ -0,0 +1,124 @@
# PRDIoT 卡风险状态限制 / 导入任务操作人与批量失效 / 凭证多图 / 预充值备注
Status: ready-for-agent
---
## Problem Statement
运营团队在日常工作中遇到四个独立痛点:
1. **风险停机/已销户卡无法有效拦截**:运营商侧将某些卡置为"风险停机"或"已销户"状态后,系统仍允许对这些卡发起复机操作,且轮询系统仍持续轮询这些卡,浪费资源并可能产生错误数据。
2. **导入任务无操作人信息**IoT 卡导入任务和设备导入任务的列表中没有"操作人"字段,无法追溯是谁发起的批量操作。同时缺少批量失效订单套餐的操作入口,运营只能逐条手动处理。
3. **凭证只能上传一张**:后台创建订单和发起退款时,只允许上传一张凭证图片,无法满足多张凭证的业务场景(如多笔转账记录、多页合同等)。
4. **代理预充值缺少备注字段**:代理预充值操作无法附加运营备注,导致事后无法追溯充值的业务原因。
---
## Solution
1. 在复机接口和轮询系统中增加对 `gateway_extend` 字段的检查,当独立卡处于"风险停机"或"已销户"状态时,拒绝复机并停止轮询。
2. 在所有导入任务中快照操作人姓名;新增"批量失效订单套餐"导入任务,支持通过 CSV 批量将订单下的套餐置为失效状态。
3. 将订单、退款、代理预充值的凭证字段改为支持最多5张的 jsonb 数组存储。
4. 在代理预充值记录上新增 `remark` 备注字段,供运营创建时填写,并在列表/详情接口中返回。
---
## User Stories
1. 作为运营人员,当我尝试对"风险停机"状态的独立卡发起复机时,我希望系统拒绝并提示原因,以免产生无效操作。
2. 作为运营人员,当我尝试对"已销户"状态的独立卡发起复机时,我希望系统拒绝并提示原因,以免对已注销的卡发起无意义请求。
3. 作为运营人员,我希望"风险停机"和"已销户"的独立卡不再参与系统轮询,以免浪费轮询资源并产生噪音数据。
4. 作为运营人员,当轮询系统检测到某张独立卡的网关状态变为"风险停机"或"已销户"时,我希望系统自动将该卡的轮询关闭,而不需要手动干预。
5. 作为运营人员,我希望在 IoT 卡导入任务列表中看到"操作人"姓名,以便追溯每次批量导入是谁发起的。
6. 作为运营人员,我希望在设备导入任务列表中看到"操作人"姓名,以便追溯每次批量导入是谁发起的。
7. 作为运营人员,我希望通过上传 CSV 文件批量将一批订单的套餐置为失效,以便快速处理异常订单。
8. 作为运营人员我希望在创建批量失效任务时能填写备注和上传凭证最多5张以便记录操作原因。
9. 作为运营人员,我希望批量失效任务完成后能看到成功数、失败数,以及失败原因(如订单号不存在),以便核查处理结果。
10. 作为运营人员,我希望批量失效套餐只影响套餐记录本身,不触发退款和其他业务流程,让轮询系统自行根据套餐状态处理停机。
11. 作为运营人员我希望在创建后台订单时可以上传最多5张凭证以便完整记录线下付款证明。
12. 作为运营人员我希望在发起退款申请时可以上传最多5张凭证以便完整记录退款证明材料。
13. 作为运营人员,我希望在代理预充值列表中看到备注内容,以便了解每笔预充值的业务背景。
14. 作为运营人员,我希望在创建代理预充值订单时填写备注,以便记录本次充值的原因或说明。
15. 作为开放 API 调用方,当我通过 Open API 对"风险停机"或"已销户"的独立卡调用复机接口时,我希望收到明确的错误响应。
---
## Implementation Decisions
### 需求一:风险停机/已销户卡限制
- **判断字段**`tb_iot_card.gateway_extend` 原文值 = `"风险停机"``"已销户"`。新增两个常量 `GatewayCardExtendRiskStop``GatewayCardExtendCancelled`
- **判断范围**:仅 `is_standalone = true` 的独立卡(未绑定设备的卡)。绑定设备的卡不受此限制。
- **复机拦截**:在 `ManualStartCard`(后台管理员入口)和 `ResumeCard`Open API 入口)两个函数中,调用网关前先读取 DB 中已落库的 `gateway_extend`,若命中则返回业务错误,不实时查询网关。
- **轮询停止**:轮询状态处理器(`PollingCardStatusHandler`)在将新的 `gateway_extend` 写入 DB 后,检查若命中风险状态且 `is_standalone=true`,则调用现有的 `UpdatePollingStatus(false)``enable_polling` 写为 `false`,并跳过 `requeueCard`,终止该卡的轮询循环。
- **错误码**:复机被拒绝时返回业务错误,提示信息明确说明原因("风险停机"/"已销户")。
### 需求二:导入任务操作人与批量失效
**操作人快照:**
- `tb_iot_card_import_task``tb_device_import_task` 表各新增 `creator_name varchar` 字段。
- 新的批量失效任务表同样包含此字段。
- 创建任务时从当前登录用户快照姓名写入,旧数据留空(不回填)。
- 列表响应 DTO 新增 `creator_name` 字段。
**批量失效订单套餐任务:**
- 新建模型、Store、Service、Handler参照现有 IoT 卡导入任务的结构CSV 上传 → 异步任务处理)。
- CSV 格式:单列 `order_no`(无表头行约定沿用现有导入任务的处理方式)。
- 创建接口额外支持:`remark`字符串备注和最多5张凭证jsonb 存储)。
- 处理逻辑:按行读取 `order_no` → 查询 `Order` → 找到 `PackageUsage WHERE order_id = ? AND status NOT IN (3, 4)` → 批量更新 `status = 4`(已失效)。
- 订单不存在记为失败fail继续处理后续行。
- 订单存在但旗下套餐全部已是终态status 3 或 4记为成功success
- 不触发退款,不直接操作 IoT 卡停机,由轮询系统根据套餐状态自行评估。
- 任务表命名:`tb_order_package_invalidate_task`
### 需求三:凭证多图改造
- `tb_order.payment_voucher_key`varchar 500`jsonb`,存储 `[]string`最多5个元素。
- `tb_refund.refund_voucher_key`varchar 500`jsonb`,存储 `[]string`最多5个元素。
- `tb_agent_recharge_record.payment_voucher_key`varchar 500`jsonb`,存储 `[]string`最多5个元素。
- **数据迁移**(同一 Migration 语句):非空旧值 `"some_key"``["some_key"]`;空字符串 → `[]`。生产环境由运营手动执行 Migration。
- 所有涉及凭证的请求 DTO 字段由 `string` 改为 `[]string`validate 标签限制 `max=5`,每项 `max=500`
- 所有涉及凭证的响应 DTO 字段改为 `[]string`
- 不限制文件类型(上传 URL 获取接口层面不变,仅存储 key 列表)。
### 需求四:代理预充值备注
- `tb_agent_recharge_record` 新增 `remark text` 字段(可为空)。
- `CreateAgentRechargeRequest` 新增 `remark` 字段(`omitempty`,无最大长度强限制,建议 1000 字符内)。
- `AgentRechargeResponse` 新增 `remark` 字段(列表和详情接口均返回)。
- 创建后不可修改,无需新增修改接口。
---
## Testing Decisions
本项目禁止自动化测试。验证方式:
- **PostgreSQL MCP**:核查 `tb_iot_card.enable_polling` 在风险状态卡被轮询后是否正确置为 `false`;核查 `tb_package_usage.status` 在批量失效任务执行后是否按预期更新;核查 jsonb 字段迁移结果。
- **curl / Postman**:对风险停机卡调用复机接口,验证返回正确错误码;上传多张凭证创建订单,验证存储和返回正确;创建代理预充值时附带备注,验证列表返回。
---
## Out of Scope
- 已绑定设备的卡(`is_standalone=false`)不受"风险停机/已销户"限制约束。
- 批量失效订单套餐不触发退款流程。
- 批量失效不直接调用网关停机,依赖轮询系统自行评估。
- 历史导入任务的 `creator_name` 不回填。
- 凭证文件类型校验(文件类型限制不在本次范围内)。
- 代理预充值备注创建后不可修改(无修改接口)。
---
## Further Notes
- 凭证字段改为 jsonb 后,前端需同步更新上传组件支持多选;本 PRD 只覆盖后端改造。
- 批量失效任务的凭证和备注字段,与代理预充值备注字段、以及需求三的多凭证设计保持一致的数据形态,便于后续统一处理。
- "风险停机"和"已销户"的 `gateway_extend` 原文值来自上游网关,若上游修改返回值需同步更新常量。

View File

@@ -0,0 +1,28 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
新增两个 gateway_extend 常量(`GatewayCardExtendRiskStop = "风险停机"``GatewayCardExtendCancelled = "已销户"`),然后在后台复机入口(`ManualStartCard`)和 Open API 复机入口(`ResumeCard`)中增加前置拦截:
- 仅对 `is_standalone = true` 的独立卡生效
- 读取 DB 中已落库的 `gateway_extend` 字段(不实时查网关)
- 若命中 `"风险停机"``"已销户"`,立即返回业务错误,不再继续调用网关
- 错误提示需明确说明被拒绝的原因(如"该卡已被运营商风险停机,不允许复机"
无 Schema 变更,不需要 Migration。
## Acceptance criteria
- [ ] `pkg/constants/iot.go` 新增 `GatewayCardExtendRiskStop``GatewayCardExtendCancelled` 两个常量
- [ ] 后台 `ManualStartCard` 在调用网关前检查独立卡 `gateway_extend`,命中风险值时返回业务错误
- [ ] Open API `ResumeCard` 在调用网关前检查独立卡 `gateway_extend`,命中风险值时返回业务错误
- [ ] 已绑定设备的卡(`is_standalone = false`)不受此限制,复机正常进行
- [ ] 返回的错误信息能区分"风险停机"和"已销户"两种情况
## Blocked by
None - can start immediately

View File

@@ -0,0 +1,28 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
`PollingCardStatusHandler` 中,当轮询查询网关后将新的 `gateway_extend` 写入 DB若新值命中风险常量`GatewayCardExtendRiskStop``GatewayCardExtendCancelled`)且该卡 `is_standalone = true`,则:
1. 调用现有的 `UpdatePollingStatus(false)``enable_polling` 写为 `false`
2. 跳过末尾的 `requeueCard`,终止该卡的轮询循环
检查时机:在 `UpdateFields` 写入 `gateway_extend` 成功之后,`requeueCard` 调用之前。
不需要 Schema 变更,不需要 Migration。
## Acceptance criteria
- [ ] 轮询处理器在写入 `gateway_extend` 后检查风险状态
- [ ] 独立卡(`is_standalone=true`)命中风险值时调用 `UpdatePollingStatus(false)``tb_iot_card.enable_polling` 被置为 `false`
- [ ] 该卡不再入队(不调用 `requeueCard`),轮询循环终止
- [ ] 已绑定设备的卡(`is_standalone=false`)命中风险值时不触发此逻辑,轮询正常继续
- [ ] 可通过 PostgreSQL MCP 验证:风险状态卡处理后 `enable_polling = false`
## Blocked by
- `issues/01-risk-resume-block.md`(依赖 `GatewayCardExtendRiskStop``GatewayCardExtendCancelled` 常量)

View File

@@ -0,0 +1,30 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
在现有两个导入任务表中新增操作人快照字段,并在列表接口返回:
- `tb_iot_card_import_task` 新增 `creator_name varchar(100)` 字段
- `tb_device_import_task` 新增 `creator_name varchar(100)` 字段
- 创建任务时从当前登录用户快照姓名写入(不可为 null旧数据留空字符串
- IoT 卡导入任务列表响应 DTO 新增 `creator_name` 字段
- 设备导入任务列表响应 DTO 新增 `creator_name` 字段
历史数据不回填,旧记录 `creator_name` 为空字符串。
## Acceptance criteria
- [ ] Migration 为两个现有任务表各添加 `creator_name varchar(100) not null default ''` 字段
- [ ] IoT 卡导入任务创建时写入当前用户姓名快照
- [ ] 设备导入任务创建时写入当前用户姓名快照
- [ ] IoT 卡导入任务列表接口返回 `creator_name` 字段
- [ ] 设备导入任务列表接口返回 `creator_name` 字段
- [ ] 旧记录 `creator_name` 为空字符串,不报错
## Blocked by
None - can start immediately

View File

@@ -0,0 +1,43 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
新增"批量失效订单套餐"导入任务,完整实现 Model → Store → Service → Handler → Route → Worker 全链路,参照现有 IoT 卡导入任务的结构。
**数据模型**(新建 `tb_order_package_invalidate_task`
- 参照 `tb_iot_card_import_task` 的基础字段(任务编号、状态、计数、失败/跳过明细、文件存储 key、时间戳等
- 额外字段:`creator_name varchar(100)``remark text``voucher_keys jsonb`(存储 `[]string`最多5个元素
**上传接口**(创建任务):
- 接受 CSV 文件(单列 `order_no`
- 额外参数:`remark`(字符串,可选)和 `voucher_keys``[]string`最多5个可选
- 写入 `creator_name` 快照
**Worker 处理逻辑**
- 逐行读取 `order_no`
-`order_no` 查询 `tb_order` 获取 `order_id`;找不到则记为失败,继续处理
- 查询 `tb_package_usage WHERE order_id = ? AND status NOT IN (3, 4)`,批量更新 `status = 4`(已失效)
- 订单存在但套餐全部已是终态status 3 或 4记为成功
- 不触发退款,不调用网关,不直接停机
**列表/详情接口**:参照现有导入任务接口结构。
## Acceptance criteria
- [ ] Migration 创建 `tb_order_package_invalidate_task` 表,包含所有必要字段
- [ ] 创建任务接口:接受 CSV + remark + voucher_keys最多5个写入 creator_name 快照
- [ ] Worker 按行处理:`order_no` 不存在 → 失败并记录原因,继续下一行
- [ ] Worker 按行处理:订单存在且有可失效套餐 → 批量更新 `status=4` → 记为成功
- [ ] Worker 按行处理:订单存在但套餐全为终态 → 记为成功
- [ ] 处理过程中不调用退款接口、不操作网关
- [ ] 列表接口返回 `creator_name``remark`、任务状态和计数
- [ ] 详情接口返回失败明细(含 order_no 和原因)
- [ ] 已在路由和文档生成器中注册
## Blocked by
None - can start immediately

View File

@@ -0,0 +1,36 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
一次 Migration 完成三个现有表的凭证字段类型变更,并同时新增代理预充值备注字段:
**字段类型变更**varchar(500) → jsonb
- `tb_order.payment_voucher_key`
- `tb_refund.refund_voucher_key`
- `tb_agent_recharge_record.payment_voucher_key`
**数据转换规则**(在同一 Migration 的 USING 子句中完成):
- 非空旧值 `"some_key"``["some_key"]`
- 空字符串 `""``[]`
- NULL → `[]`
**新增字段**
- `tb_agent_recharge_record.remark text`(可为空,默认 null
生产环境由运营手动执行Migration 文件需包含完整的 UP 和 DOWN 语句。
## Acceptance criteria
- [ ] Migration UP三个凭证字段成功从 varchar 转为 jsonb历史单值数据转换为单元素数组
- [ ] Migration UP空字符串/NULL 转换为空数组 `[]`
- [ ] Migration UP`tb_agent_recharge_record` 新增 `remark text` 字段
- [ ] Migration DOWN可回滚jsonb → varchar取数组第一个元素remark 字段删除)
- [ ] 迁移后可用 PostgreSQL MCP 验证数据格式正确
## Blocked by
None - can start immediately

View File

@@ -0,0 +1,27 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
将后台创建订单接口的凭证字段从单个字符串改为最多5个字符串的列表
- `CreateAdminOrderRequest.payment_voucher_key``string``[]string`validate 限制 `max=5`,每项 `max=500`
- `OrderResponse.payment_voucher_key`(以及所有订单相关响应 DTO`string``[]string`
- Order Model 的 `PaymentVoucherKey` 字段类型改为适配 jsonb 的 `pq.StringArray` 或自定义 JSON 类型
- Order Service 中读写 `payment_voucher_key` 的逻辑同步更新
Schema 变更已由 `issues/05-voucher-multi-migration.md` 完成。
## Acceptance criteria
- [ ] 后台创建订单接口接受 `payment_voucher_key []string`0-5个元素offline 支付时至少需要1个
- [ ] 订单列表/详情接口返回 `payment_voucher_key []string`
- [ ] 传入超过5个凭证 key 时返回参数校验错误
- [ ] 现有无凭证的订单wallet 支付)返回空数组,不报错
## Blocked by
- `issues/05-voucher-multi-migration.md`

View File

@@ -0,0 +1,29 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
将退款申请接口的凭证字段从单个字符串改为最多5个字符串的列表
- `CreateRefundRequest.refund_voucher_key``string``[]string`validate 限制 `max=5`,每项 `max=500`required至少1个
- `ResubmitRefundRequest.refund_voucher_key``*string``*[]string`(重新提交时可选替换)
- `RefundResponse.refund_voucher_key`(以及所有退款相关响应 DTO`string``[]string`
- Refund Model 的 `RefundVoucherKey` 字段类型改为适配 jsonb 的类型
- Refund Service 中读写 `refund_voucher_key` 的逻辑同步更新
Schema 变更已由 `issues/05-voucher-multi-migration.md` 完成。
## Acceptance criteria
- [ ] 创建退款申请接口接受 `refund_voucher_key []string`1-5个元素必填
- [ ] 重新提交退款接口的 `refund_voucher_key` 可选,传入时替换全部凭证
- [ ] 退款列表/详情接口返回 `refund_voucher_key []string`
- [ ] 传入超过5个凭证 key 时返回参数校验错误
- [ ] 传入空数组时返回参数校验错误至少需要1个
## Blocked by
- `issues/05-voucher-multi-migration.md`

View File

@@ -0,0 +1,35 @@
Status: done
## Parent
`.scratch/iot-risk-import-voucher-remark/PRD.md`
## What to build
同时完成代理预充值的凭证多图改造和备注字段接入:
**凭证多图**
- `CreateAgentRechargeRequest.payment_voucher_key``string``[]string`validate 限制 `max=5`,每项 `max=500`offline 支付时至少1个微信支付时忽略
- `AgentRechargeResponse.payment_voucher_key``string``[]string`
- `AgentRechargeRecord.PaymentVoucherKey` Model 字段类型改为适配 jsonb 的类型
**备注字段**
- `CreateAgentRechargeRequest` 新增 `remark string`可选建议不超过1000字符
- `AgentRechargeResponse` 新增 `remark string`(列表和详情接口均返回)
- `AgentRechargeRecord.Remark` 在创建时写入,后续不可修改
AgentRecharge Service 中读写相关字段的逻辑同步更新。
Schema 变更payment_voucher_key 类型 + remark 字段)已由 `issues/05-voucher-multi-migration.md` 完成。
## Acceptance criteria
- [ ] 创建代理预充值接口接受 `payment_voucher_key []string`offline 时至少1个和可选 `remark`
- [ ] 代理预充值列表/详情接口返回 `payment_voucher_key []string``remark`
- [ ] 传入超过5个凭证 key 时返回参数校验错误
- [ ] 不新增 remark 修改接口,创建后备注字段只读
- [ ] 微信支付创建预充值时可不传凭证(与现有逻辑一致)
## Blocked by
- `issues/05-voucher-multi-migration.md`

View File

@@ -346,6 +346,19 @@ func startPollingScheduler(
appLogger,
)
// 注入套餐失效前流量同步器:最小化 iot_card.Service只需 gateway + store + redis + 流量扣减)
trafficSyncer := iot_card_svc.New(
runtime.db,
runtime.pollingIotCardStore,
nil, nil, nil, nil, nil,
runtime.gatewayClient,
appLogger,
nil,
)
trafficSyncer.SetRedisClient(runtime.redisClient)
trafficSyncer.SetDataDeductor(runtime.workerResult.Services.UsageService)
activationHandler.SetTrafficSyncer(trafficSyncer)
pollingScheduler := polling.NewScheduler(
runtime.redisClient,
runtime.asynqClient,

343
docs/改造方向.md Normal file
View File

@@ -0,0 +1,343 @@
# 系统改造方向
> 本文档记录当前系统的核心问题、改造原因、改造方向和预期收益。
> 用于在开发过程中明确"干什么、为什么干、干的好处"。
---
## 一、现状
### 1.1 项目规模
- 当前在线卡数:约 3 万张(生产)
- 待迁移卡数:约 67 万张(来自旧系统)
- 核心业务链路:代理购卡 → 销售分配 → 套餐购买 → 流量同步 → 停复机
### 1.2 轮询系统现状
轮询系统的作用是**定期向运营商查询卡的状态**(实名状态、流量用量、卡网络状态等),保持系统数据与运营商同步。
当前架构:
```
全量卡 ID → Redis Sorted Set分片score = 下次检查时间)
→ Scheduler 每秒扫描到期卡
→ Asynq 队列
→ Worker 执行检查(实名/流量/卡状态/保护期/套餐,共 5 种类型)
```
**问题清单:**
| # | 问题 | 具体表现 |
|---|------|---------|
| 1 | **看不见,无法排查** | 业务反馈"某张卡一小时了还没同步实名",没有任何结构化记录可以查,只能翻日志文件靠关键词过滤 |
| 2 | **配置改了不知道有没有用** | 轮询配置对哪些卡生效完全黑盒,改完后没有验证手段,全靠等和猜 |
| 3 | **大量无意义轮询** | 全量卡按固定频率轮询大多数卡状态根本没有变化这些查询全是浪费67 万张卡迁移后压力更大 |
| 4 | **业务事件不触发检查** | 代理通过开放接口买了套餐,系统不知道,要等下次定时轮询碰到这张卡(可能要等几分钟到几十分钟)|
| 5 | **两种不同职责混在一个调度器** | 卡状态同步(每 1 秒触发)和业务流程调度(套餐过期处理、流量重置,每 10 秒触发)混在同一个 Scheduler出问题难以定位是哪一层 |
| 6 | **调度状态全在 Redis没有持久化** | Redis 重启或清空后全量卡的调度状态丢失需要重跑全量初始化才能恢复67 万张卡的初始化需要相当长时间,期间所有卡停止轮询 |
| 7 | **单卡状态散在四处** | 上次检查时间在 IotCard 表;下次检查时间在 Redis Sorted Set卡状态快照在 Redis Hash执行结果在日志文件。没有一个地方能给完整答案 |
### 1.3 流量数据现状
运营商接口返回的是**当前账期内的累计已用流量**(不是增量,是总量)。
运营商账期重置日因运营商而异,在创建运营商时配置。
套餐周期可以是自然月或固定日期,在创建套餐时决定。
当前 IotCard 表上与流量相关的字段:
```
DataUsageMB 卡生命周期总用量(永不归零)
CurrentMonthUsageMB 系统自然月累计用量(展示用)
LastMonthTotalMB 上月结束时的流量总量(用于跨月计算)
CurrentMonthStartDate 系统自然月起始日期
LastGatewayReadingMB 上次轮询时运营商返回的累计读数
```
PackageUsage 表上:
```
DataUsageMB 套餐已用量(套餐周期到期时归零,用于判断是否耗尽)
```
**问题清单:**
| # | 问题 | 具体表现 |
|---|------|---------|
| 1 | **运营商重置靠猜,有漏检风险** | 只存上一次读数(`LastGatewayReadingMB`),靠"本次读数比上次小"来猜测运营商是否重置了账期。若轮询有空窗(例如系统故障几小时),重置后的新用量超过了上次读数,则增量为正,系统判断为"没有重置",漏掉了那段用量 |
| 2 | **无原始读数,无法审计** | 运营商告诉了什么、什么时候告诉的、算出了多少增量——计算完成后全部丢弃。流量统计不对时,没有任何数据可以追溯原因 |
| 3 | **重置边界的流量归属不准** | 套餐在午夜重置,但上次轮询在 23:59下次轮询在 00:01。这两分钟的增量包含昨天剩余用量全部算进今天日流量套餐受影响明显 |
| 4 | **三套"月"的概念同时存在** | 运营商账期(各自定义重置日)+ 套餐周期(自然月或固定日)+ 系统自然月(展示用)。三者不对齐,边界处理散落在代码各处,逻辑复杂且脆弱 |
| 5 | **`CurrentMonthUsageMB` 不按时重置** | 这个字段不由重置任务触发而是在下次轮询时内联检测跨月并重置。1 号凌晨用户看到的数据是上月的,要等到下次轮询才更新 |
| 6 | **两套日流量记录并存** | `CardDailyUsage`(按卡)和 `PackageUsageDailyRecord`(按套餐)同时存在,来源不同,权威性不明确,出现差异时不知道信谁 |
### 1.4 整体架构现状
当前分层:`Handler → Service → Store → Model`,分层存在但执行不彻底。
核心问题:**贫血模型 + 上帝 Service**
- Model 只是数据库映射,没有任何业务行为
- 所有业务逻辑全部堆在 Service导致 Service 极度臃肿
| Service | 行数 | 混杂的业务域 |
|---------|------|------------|
| order/service.go | 3031 行 | 订单创建、支付、佣金计算、代购、库存扣减 |
| iot_card/service.go | 2310 行 | 卡管理、网关调用、停复机、生命周期、流量同步 |
| device/service.go | 2078 行 | 设备管理、绑定、批量导入 |
没有领域事件:业务事件(套餐支付完成、卡实名成功)不产生任何可订阅的事件,所有下游动作全部通过直接方法调用耦合,新增联动逻辑必须修改原有代码。
---
## 二、改造方向(优先级一):轮询系统 + 流量数据
### 2.1 轮询系统改造
#### 改造 A建立单卡可查询的轮询状态
**要干什么:** 新增一张表 `card_polling_state`,每张卡一行,只更新不追加,记录每种检查类型的上次执行时间、结果、命中配置、下次预计时间。
**为什么干:** 现在"这张卡的轮询状态"散在四个地方,没有一处能完整回答问题。这张表就是单一的事实来源。
**好处:**
- 业务反馈某张卡没同步 → 查这张表30 秒给出答案(上次检查时间、结果是什么、下次什么时候)
- 不影响任何现有轮询逻辑,只需在执行检查后多写一次 UPDATE
**同时新增两个接口:**
```
GET /admin/polling/cards/:id/state
→ 返回这张卡的完整轮询状态
GET /admin/polling/preview?card_id=12345
→ 返回这张卡当前命中哪条配置、各检查类型的间隔是多少、下次检查时间
```
第二个接口解决"改了配置不知道有没有用"的问题——改完配置后,查任意一张卡,立刻知道新配置有没有生效。
---
#### 改造 B引入事件触发层
**要干什么:** 在业务动作完成时,主动安排针对性的延迟检查,而不是等定时轮询碰到这张卡。
**为什么干:** 现在买了套餐,系统要等下次轮询才知道套餐有没有激活。运营商激活需要时间,但我们不知道什么时候激活完成,只能等碰到。
**好处:**
- 套餐购买 → 5 分钟后主动检查 → 激活状态及时同步,不用等下次定时轮询
- 轮询有了"意义"——有事件发生才检查,而不是盲目扫全量
**触发规则:**
| 业务事件 | 触发检查 | 延迟 |
|---------|---------|------|
| 套餐订单支付完成 | 检查套餐激活状态 | 5 分钟 |
| 代理开放接口购套餐 | 检查套餐激活状态 | 5 分钟 |
| 卡首次分配给店铺 | 检查实名状态 | 30 分钟 |
| 手动触发刷新 | 立即检查 | 0 |
实现方式:利用 Asynq 的定时任务功能(`asynq.ProcessAt`),在现有 Worker 框架内完成,不需要新增基础设施。
---
#### 改造 C按卡状态分级调度
**要干什么:** 不同状态的卡,轮询频率不同,而不是所有卡一视同仁。
**为什么干:** 稳定运行了半年、没有套餐变动的卡,每 2 分钟轮询一次是纯粹的浪费。等实名的卡、刚购套餐的卡才需要频繁检查。
**好处:**
- 67 万张卡迁移后,无意义轮询量大幅下降(估计减少 60%-80%
- 轮询资源集中在真正需要关注的卡
**分级规则(通过现有 PollingConfig 机制配置):**
| 卡状态 | 实名检查间隔 | 流量检查间隔 |
|-------|------------|------------|
| 等待实名 | 10 分钟 | 30 分钟 |
| 活跃中(近期有套餐变动)| 15 分钟 | 30 分钟 |
| 稳定运行 | 2 小时 | 2 小时 |
| 停机 / 停用 | 每日一次 | 每日一次 |
---
#### 改造 D分离业务调度
**要干什么:** 把套餐过期处理、流量重置从 `Scheduler` 里拆出去,用独立的定时任务管理。
**为什么干:** 现在卡状态同步(每 1 秒)和业务逻辑调度(每 10 秒)混在同一个 Scheduler 里。出了问题不知道是哪一层,改动也容易互相影响。
**好处:**
- 各司其职,互不干扰
- 业务调度(套餐过期)出问题不影响卡状态同步
- 代码更清晰,新增调度任务不需要修改 Scheduler
---
### 2.2 流量数据改造
#### 改造 A保留原始读数
**要干什么:** 新增 `carrier_readings` 表,每次从运营商拿到读数都记录下来,永不修改。
```
carrier_readings:
card_id 卡ID
reading_mb 运营商返回的当期累计值
read_at 读取时间
cycle_start_date 本条读数所属的运营商账期开始日期
source 来源polling / manual / triggered
```
**为什么干:** 现在计算完增量后,原始读数就丢弃了。流量统计不对时,没有任何数据可以追溯。
**好处:**
- 流量统计有问题 → 查这张表,还原每次拿到了什么数据、算出了多少增量
- 不再是"只有最终结果,不知道过程"
- 可以离线重算历史数据,修复过去的错误
**存储量控制:** 保留 30 天原始记录,超过 30 天只保留"状态变化"事件(数量极少)。分区表实现,不影响查询性能。
---
#### 改造 B基于账期的重置检测
**要干什么:**`cycle_start_date` 字段判断运营商是否重置了账期,替代现在靠"增量为负"来猜测的方式。
**为什么干:** 现在的猜测方式有漏检风险——如果轮询空窗期间运营商重置且新用量超过了旧读数,增量为正,系统无法检测到重置,漏计那段用量。
**好处:**
- 重置检测从"猜"变成"判断",准确率 100%
- 彻底消除因轮询空窗导致的流量漏计
- 代码逻辑更简单,不再需要"重置窗口"时间判断
```go
// 改前:靠负增量猜测,有漏检风险
if increment < 0 && isRefreshResetWindow(now, resetDay) { ... }
// 改后:直接判断账期,准确无误
if prev.CycleStartDate != curr.CycleStartDate {
increment = curr.ReadingMB // 不同账期 = 运营商重置,当前读数即为增量
}
```
---
#### 改造 C清理 IotCard 上的流量计算字段
**要干什么:**`LastGatewayReadingMB``CurrentMonthStartDate``LastMonthTotalMB` 从 IotCard 表移除,这些信息改由 `carrier_readings` 表承载。
IotCard 只保留两个对外展示的汇总值:
- `CurrentMonthUsageMB`:展示用,由定时任务按自然月计算并主动更新(不再等轮询时内联检测)
- `DataUsageMB`:生命周期总量,由增量事件累加
**为什么干:** 现在 IotCard 同时承担了三个角色:卡的身份信息、流量计数器、流量计算中间状态。职责太多,一张表改动牵一发动全身。
**好处:**
- IotCard 回归本职:描述这张卡是谁、状态是什么
- 流量计算有了独立的数据层,不污染主体数据
- `CurrentMonthUsageMB` 在月初会被主动更新,不再有"1号凌晨数据是上月的"问题
---
## 三、改造方向优先级二DDD 整体方向
> 这部分是更长期的方向,在优先级一完成后推进。
### 3.1 核心思想
**积木本身是安全的,拼积木组成业务流程。**
- **积木** = 领域对象(`IotCard``Order``Wallet`):自己知道自己的规则,外部只能通过方法操作,不能随意改字段
- **拼积木** = 应用服务(用例):只负责"拿数据 → 调对象方法 → 保存 → 发事件",不含业务判断逻辑
**AI 辅助开发的约束价值:** 有了这个结构AI 生成代码时只能往"一个用例一个文件"的模式里填,不会生成新的上帝 Service业务规则在领域对象里AI 调用对象方法时规则自动生效。
### 3.2 目标目录结构
```
internal/
├── domain/ 领域层(纯业务逻辑,不依赖任何框架)
│ ├── order/
│ │ ├── order.go 聚合根Order 实体有方法Pay, Cancel, Reject
│ │ ├── events.go 领域事件OrderPaid, OrderCancelled
│ │ ├── repository.go 仓储接口interface不含实现
│ │ └── values.go 值对象OrderStatus, Amount
│ ├── asset/
│ │ ├── iot_card.go IotCard 聚合根
│ │ └── repository.go
│ └── wallet/
│ └── wallet.go Wallet 聚合根Debit, Credit
├── application/ 应用层(用例,只做编排)
│ ├── order/
│ │ ├── create_order.go 一个文件一个用例
│ │ ├── pay_order.go
│ │ └── cancel_order.go
│ └── asset/
├── infrastructure/ 基础设施层(实现接口)
│ ├── persistence/ 原来的 store/postgres/
│ └── event/ 事件总线
└── interface/ 接口层
├── http/ 原来的 handler/ + routes/
└── task/ Asynq task handlers
```
### 3.3 迁移策略(绞杀者模式)
不全量重写,新旧并行,逐步替换:
1. 建好 `domain/``application/` 目录和规范文件
2. **所有新功能按新结构写**,不往旧 Service 里加代码
3. 最痛的模块优先迁移:`order`3031 行)和 `iot_card`2310 行)
4. 每次迁移一个用例,验证跑通后再迁移下一个
---
## 四、实施顺序
```
第一阶段(当前)
├── 新增 card_polling_state 表(轮询状态持久化)
├── 新增轮询状态查询接口 + 配置预览接口(可观测性)
├── 新增 carrier_readings 表(保留原始读数)
├── 改造增量计算逻辑(基于账期判断重置)
└── 事件触发层(套餐购买 → 延迟检查)
第二阶段
├── 分级调度(按卡状态动态调整轮询频率)
├── 分离业务调度(套餐过期/流量重置从 Scheduler 独立)
└── 清理 IotCard 流量计算字段
第三阶段DDD 改造)
├── 建立目录骨架和规范示例
├── 新功能按新结构写
└── 逐步迁移最痛的模块
```
---
## 五、预期收益
### 轮询 + 流量改造后
| 场景 | 改造前 | 改造后 |
|------|--------|--------|
| 业务反馈"某张卡没同步" | 翻日志,靠运气,可能花几十分钟 | 查接口30 秒内给出答案 |
| 验证配置是否生效 | 完全黑盒,改完只能等和猜 | 配置预览接口,改完立即验证 |
| 流量统计不对 | 无法追溯,无从下手 | 查原始读数,还原每次增量计算 |
| 运营商重置漏计 | 有风险,轮询空窗时必现 | 账期判断100% 准确 |
| 套餐购买后同步延迟 | 等下次定时轮询,最多几十分钟 | 5 分钟内主动检查 |
| 67 万卡迁移后轮询压力 | 全量扫描,压力线性增长 | 分级 + 事件触发,压力可控 |
### DDD 改造后
| 场景 | 改造前 | 改造后 |
|------|--------|--------|
| AI 辅助开发新功能 | 容易生成新的上帝 Service | 遵循用例模式,每次只加一个文件 |
| 新增业务联动逻辑 | 必须修改原有 Service 代码 | 新增事件监听器,原有代码不动 |
| 定位业务规则在哪里 | 散在几千行 Service 里找 | 在对应的领域对象方法里 |
| 排查某个流程的问题 | 跨多个 Service 追调用链 | 用例文件即流程,一目了然 |

View File

@@ -141,7 +141,7 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
AgentRecharge: admin.NewAgentRechargeHandler(svc.AgentRecharge, validate),
Refund: admin.NewRefundHandler(svc.Refund),
OrderPackageInvalidate: admin.NewOrderPackageInvalidateHandler(svc.OrderPackageInvalidate),
ClientWechat: app.NewClientWechatHandler(svc.WechatConfig, deps.Redis, deps.Logger),
ClientWechat: app.NewClientWechatHandler(svc.WechatConfig, deps.Redis, deps.Logger),
SuperAdmin: admin.NewSuperAdminHandler(svc.OperationPassword),
AgentOpenAPI: openapiHandler.NewHandler(svc.AgentOpenAPI, validate),
}

View File

@@ -74,6 +74,29 @@ func (h *AgentRechargeHandler) Get(c *fiber.Ctx) error {
return response.Success(c, result)
}
// Reject 驳回代理充值订单
// POST /api/admin/agent-recharges/:id/reject
func (h *AgentRechargeHandler) Reject(c *fiber.Ctx) error {
id, err := strconv.ParseUint(c.Params("id"), 10, 64)
if err != nil {
return errors.New(errors.CodeInvalidParam, "无效的充值记录ID")
}
var req dto.AgentRechargeRejectRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validator.Struct(&req); err != nil {
return errors.New(errors.CodeInvalidParam)
}
if err := h.service.Reject(c.UserContext(), uint(id), req.RejectionReason); err != nil {
return err
}
return response.Success(c, nil)
}
// OfflinePay 确认线下充值
// POST /api/admin/agent-recharges/:id/offline-pay
func (h *AgentRechargeHandler) OfflinePay(c *fiber.Ctx) error {

View File

@@ -56,12 +56,13 @@ func (h *AssetHandler) SetLifecycleService(svc AssetLifecycleService) {
}
// resolveByIdentifier 通过标识符解析资产,用于各处理器的公共逻辑
// 不传递 include_usage_summary始终为 false不影响其他 Handler 性能
func (h *AssetHandler) resolveByIdentifier(c *fiber.Ctx) (*dto.AssetResolveResponse, error) {
identifier := c.Params("identifier")
if identifier == "" {
return nil, errors.New(errors.CodeInvalidParam, "标识符不能为空")
}
return h.assetService.Resolve(c.UserContext(), identifier)
return h.assetService.Resolve(c.UserContext(), identifier, false)
}
// Resolve 通过任意标识符解析资产(设备或卡)
@@ -72,7 +73,8 @@ func (h *AssetHandler) Resolve(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "标识符不能为空")
}
result, err := h.assetService.Resolve(c.UserContext(), identifier)
includeUsageSummary := c.QueryBool("include_usage_summary", false)
result, err := h.assetService.Resolve(c.UserContext(), identifier, includeUsageSummary)
if err != nil {
return err
}
@@ -378,7 +380,7 @@ func (h *AssetHandler) UpdatePollingStatus(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam)
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -424,7 +426,7 @@ func (h *AssetHandler) UpdateRealnamePolicy(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "无效的实名认证策略")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -467,7 +469,7 @@ func (h *AssetHandler) UpdateRealnameStatus(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "无效的实名状态")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}

View File

@@ -46,7 +46,7 @@ func (h *AssetWalletHandler) GetWallet(c *fiber.Ctx) error {
return errors.New(errors.CodeInternalError, "资产服务未配置")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -76,7 +76,7 @@ func (h *AssetWalletHandler) ListTransactions(c *fiber.Ctx) error {
return errors.New(errors.CodeInternalError, "资产服务未配置")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}

View File

@@ -98,7 +98,7 @@ func (h *ClientAssetHandler) resolveAssetFromIdentifier(c *fiber.Ctx, identifier
}
skipPermissionCtx := context.WithValue(c.UserContext(), constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier)
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -69,7 +69,7 @@ func (h *ClientDeviceHandler) validateDeviceAsset(c *fiber.Ctx, identifier strin
ctx := c.UserContext()
// 通过标识符解析资产
asset, err := h.assetService.Resolve(ctx, identifier)
asset, err := h.assetService.Resolve(ctx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -84,7 +84,7 @@ func (h *ClientRealnameHandler) GetRealnameLink(c *fiber.Ctx) error {
ctx := c.UserContext()
// 3. 通过标识符解析资产
asset, err := h.assetService.Resolve(ctx, req.Identifier)
asset, err := h.assetService.Resolve(ctx, req.Identifier, false)
if err != nil {
return err
}

View File

@@ -279,6 +279,18 @@ func (h *ClientWalletHandler) CreateRecharge(c *fiber.Ctx) error {
return err
}
// 第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// switch resolved.Asset.AssetType {
// case "card":
// if req.PaymentMethod != constants.RechargeMethodAlipay {
// return errors.New(errors.CodeInvalidParam, "单卡资产只支持支付宝充值")
// }
// case "device":
// if req.PaymentMethod != constants.RechargeMethodWechat {
// return errors.New(errors.CodeInvalidParam, "设备资产只支持微信充值")
// }
// }
config, err := h.wechatConfigService.GetActiveConfig(resolved.SkipPermissionCtx)
if err != nil {
return err
@@ -540,7 +552,7 @@ func (h *ClientWalletHandler) resolveAssetFromIdentifier(c *fiber.Ctx, identifie
}
skipPermissionCtx := context.WithValue(c.UserContext(), constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier)
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -84,6 +84,7 @@ type AgentRechargeRecord struct {
Status int `gorm:"column:status;type:int;not null;default:1;comment:充值状态(1-待支付 2-已支付 3-已完成 4-已关闭 5-已退款)" json:"status"`
PaymentVoucherKey StringJSONBArray `gorm:"column:payment_voucher_key;type:jsonb;comment:支付凭证对象存储Key列表线下支付时必填最多5个微信支付时为空" json:"payment_voucher_key"`
Remark string `gorm:"column:remark;type:text;comment:运营备注(创建时填写,不可修改)" json:"remark,omitempty"`
RejectionReason *string `gorm:"column:rejection_reason;type:varchar(500);comment:驳回原因,仅驳回时写入" json:"rejection_reason,omitempty"`
PaidAt *time.Time `gorm:"column:paid_at;comment:支付时间" json:"paid_at,omitempty"`
CompletedAt *time.Time `gorm:"column:completed_at;comment:完成时间" json:"completed_at,omitempty"`
ShopIDTag uint `gorm:"column:shop_id_tag;not null;index;comment:店铺ID标签(多租户过滤)" json:"shop_id_tag"`

View File

@@ -106,7 +106,7 @@ type AgentOpenAPIWalletTransactionListRequest struct {
// AgentOpenAPIWalletPackageOrderRequest 开放接口钱包套餐购买请求
type AgentOpenAPIWalletPackageOrderRequest struct {
CardNos []string `json:"card_nos" validate:"required,min=1,max=100,dive,min=1,max=100" required:"true" minItems:"1" maxItems:"100" description:"卡标识列表(支持 ICCID、虚拟号、MSISDN"`
CardNos []string `json:"card_nos" validate:"required,min=1,max=100,dive,min=1,max=100" required:"true" minItems:"1" maxItems:"100" description:"卡标识列表(支持 ICCID、虚拟号、MSISDN、设备 IMEI传入设备 IMEI 时自动转为设备维度购买"`
PackageCode string `json:"package_code" validate:"required,min=1,max=100" required:"true" minLength:"1" maxLength:"100" description:"套餐编码(一次只能购买一个套餐)"`
}

View File

@@ -22,24 +22,36 @@ type AgentOfflinePayParams struct {
// AgentRechargeResponse 代理充值记录响应
type AgentRechargeResponse struct {
ID uint `json:"id" description:"充值记录ID"`
RechargeNo string `json:"recharge_no" description:"充值单号(ARCH前缀)"`
ShopID uint `json:"shop_id" description:"店铺ID"`
ShopName string `json:"shop_name" description:"店铺名称"`
AgentWalletID uint `json:"agent_wallet_id" description:"代理钱包ID"`
Amount int64 `json:"amount" description:"充值金额(分)"`
PaymentMethod string `json:"payment_method" description:"支付方式 (wechat:微信在线支付, offline:线下转账)"`
PaymentChannel string `json:"payment_channel" description:"实际支付通道 (wechat_direct:微信直连, fuyou:富友, offline:线下转账)"`
PaymentConfigID *uint `json:"payment_config_id" description:"关联支付配置ID线下充值为null"`
PaymentTransactionID string `json:"payment_transaction_id" description:"第三方支付流水号"`
ID uint `json:"id" description:"充值记录ID"`
RechargeNo string `json:"recharge_no" description:"充值单号(ARCH前缀)"`
ShopID uint `json:"shop_id" description:"店铺ID"`
ShopName string `json:"shop_name" description:"店铺名称"`
AgentWalletID uint `json:"agent_wallet_id" description:"代理钱包ID"`
Amount int64 `json:"amount" description:"充值金额(分)"`
PaymentMethod string `json:"payment_method" description:"支付方式 (wechat:微信在线支付, offline:线下转账)"`
PaymentChannel string `json:"payment_channel" description:"实际支付通道 (wechat_direct:微信直连, fuyou:富友, offline:线下转账)"`
PaymentConfigID *uint `json:"payment_config_id" description:"关联支付配置ID线下充值为null"`
PaymentTransactionID string `json:"payment_transaction_id" description:"第三方支付流水号"`
PaymentVoucherKey []string `json:"payment_voucher_key" description:"支付凭证对象存储Key列表线下支付时存在最多5个"`
Remark string `json:"remark,omitempty" description:"运营备注"`
Status int `json:"status" description:"状态 (1:待支付, 2:已支付, 3:已完成, 4:已关闭, 5:已退款)"`
StatusName string `json:"status_name" description:"状态名称(中文)"`
PaidAt *string `json:"paid_at" description:"支付时间"`
CompletedAt *string `json:"completed_at" description:"完成时间"`
CreatedAt string `json:"created_at" description:"创建时间"`
UpdatedAt string `json:"updated_at" description:"更新时间"`
Status int `json:"status" description:"状态 (1:待支付, 2:已支付, 3:已完成, 4:已关闭, 5:已退款, 6:已驳回)"`
StatusName string `json:"status_name" description:"状态名称(中文)"`
RejectionReason *string `json:"rejection_reason,omitempty" description:"驳回原因,仅 status=6 时有值"`
PaidAt *string `json:"paid_at" description:"支付时间"`
CompletedAt *string `json:"completed_at" description:"完成时间"`
CreatedAt string `json:"created_at" description:"创建时间"`
UpdatedAt string `json:"updated_at" description:"更新时间"`
}
// AgentRechargeRejectRequest 驳回代理充值订单请求
type AgentRechargeRejectRequest struct {
RejectionReason string `json:"rejection_reason" validate:"required,max=500" required:"true" description:"驳回原因必填最多500字"`
}
// AgentRechargeRejectParams 驳回代理充值订单聚合参数(用于文档生成)
type AgentRechargeRejectParams struct {
IDReq
AgentRechargeRejectRequest
}
// AgentRechargeListRequest 代理充值记录列表请求
@@ -47,7 +59,7 @@ type AgentRechargeListRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码默认1"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页条数默认20最大100"`
ShopID *uint `json:"shop_id" query:"shop_id" description:"按店铺ID过滤"`
Status *int `json:"status" query:"status" description:"按状态过滤 (1:待支付, 2:已支付, 3:已完成, 4:已关闭, 5:已退款)"`
Status *int `json:"status" query:"status" description:"按状态过滤 (1:待支付, 2:已支付, 3:已完成, 4:已关闭, 5:已退款, 6:已驳回)"`
StartDate string `json:"start_date" query:"start_date" description:"创建时间起始日期(YYYY-MM-DD)"`
EndDate string `json:"end_date" query:"end_date" description:"创建时间截止日期(YYYY-MM-DD)"`
}

View File

@@ -68,6 +68,9 @@ type AssetResolveResponse struct {
NetworkStatus int `json:"network_status" description:"网络状态0停机 1开机asset_type=card时有效"`
GatewayExtend string `json:"gateway_extend" description:"Gateway 卡状态扩展字段,原样返回上游 extend用于展示运营商侧实际停机原因"`
GatewayCardIMEI string `json:"gateway_card_imei" description:"插拔卡业务 IMEI由 Gateway 卡状态接口同步,非设备自身 IMEI无业务含义仅供查看"`
// 流量汇总字段(仅在 ?include_usage_summary=true 时返回非 null 值)
TotalVirtualUsedMB *float64 `json:"total_virtual_used_mb" description:"当前世代所有套餐虚已用之和MB未请求时为 null"`
TotalVirtualRemainingMB *float64 `json:"total_virtual_remaining_mb" description:"当前世代所有套餐虚剩余之和MB未请求时为 null"`
}
// BoundCardInfo 设备绑定的卡信息
@@ -148,9 +151,10 @@ type AssetPackagesResult struct {
Items []*AssetPackageResponse `json:"items" description:"套餐列表"`
}
// AssetResolveRequest 资产解析请求(路径参数)
// AssetResolveRequest 资产解析请求
type AssetResolveRequest struct {
Identifier string `path:"identifier" description:"资产标识符(虚拟号/ICCID/IMEI/SN/MSISDN" required:"true"`
Identifier string `path:"identifier" description:"资产标识符(虚拟号/ICCID/IMEI/SN/MSISDN" required:"true"`
IncludeUsageSummary bool `query:"include_usage_summary" description:"是否返回当前世代流量汇总字段total_virtual_used_mb / total_virtual_remaining_mb默认 false"`
}
// AssetIdentifierRequest 资产标识符路径参数请求

View File

@@ -81,6 +81,30 @@ func (s *PollingLifecycleService) OnCardStatusChanged(ctx context.Context, cardI
s.enqueueCard(ctx, card)
}
// OnBatchCardsStatusChanged 批量卡状态变化:单次 Redis pipeline 清理 + 单次 DB 批量查询
// 替代 N 次 OnCardStatusChanged 调用,避免批量分配/回收时打满 DB 连接池
func (s *PollingLifecycleService) OnBatchCardsStatusChanged(ctx context.Context, cardIDs []uint) {
if len(cardIDs) == 0 {
return
}
// 单次 pipeline批量从当前分片队列移除并清理缓存
if err := s.queueMgr.RemoveBatchFromCurrentShardQueues(ctx, cardIDs); err != nil {
s.logger.Warn("批量卡状态变化:从队列移除失败", zap.Int("count", len(cardIDs)), zap.Error(err))
}
// 单次 DB 查询:批量加载所有卡信息
cards, err := s.iotCardStore.GetByIDs(ctx, cardIDs)
if err != nil {
s.logger.Error("批量卡状态变化:加载卡信息失败", zap.Int("count", len(cardIDs)), zap.Error(err))
return
}
for _, card := range cards {
if !s.shouldEnqueue(ctx, card) {
continue
}
s.enqueueCard(ctx, card)
}
}
// OnCardDeleted 卡删除后移除所有队列并清理缓存
func (s *PollingLifecycleService) OnCardDeleted(ctx context.Context, cardID uint) {
if err := s.queueMgr.OnCardDeleted(ctx, cardID); err != nil {

View File

@@ -17,6 +17,12 @@ import (
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
// TrafficSyncer 套餐失效前流量同步接口
// 在主套餐标记为已过期之前,从 Gateway 拉取最新流量写入 DB
type TrafficSyncer interface {
RefreshCardDataByID(ctx context.Context, cardID uint) error
}
// PackageActivationHandler 套餐激活检查处理器
// 任务 19: 处理主套餐过期、加油包级联失效、待生效主套餐激活
type PackageActivationHandler struct {
@@ -27,6 +33,7 @@ type PackageActivationHandler struct {
deviceSimBinding *postgres.DeviceSimBindingStore
activationService *packagepkg.ActivationService
stopResumeCallback packagepkg.StopResumeCallback // 停复机回调,用于套餐过期后主动停机
trafficSyncer TrafficSyncer // 套餐失效前流量同步,可选
logger *zap.Logger
}
@@ -60,6 +67,48 @@ func NewPackageActivationHandler(
}
}
// SetTrafficSyncer 注入套餐失效前流量同步器(在 Start 前调用)
func (h *PackageActivationHandler) SetTrafficSyncer(syncer TrafficSyncer) {
h.trafficSyncer = syncer
}
// syncTrafficBeforeExpiry 套餐失效前从 Gateway 同步一次最新流量
// iot_card 直接同步device 遍历所有绑定卡同步
// 失败只记录警告,不阻断失效流程
func (h *PackageActivationHandler) syncTrafficBeforeExpiry(ctx context.Context, carrierType string, carrierID uint) {
if h.trafficSyncer == nil {
return
}
if carrierType == constants.AssetTypeIotCard {
if err := h.trafficSyncer.RefreshCardDataByID(ctx, carrierID); err != nil {
h.logger.Warn("套餐失效前流量同步失败",
zap.String("carrier_type", carrierType),
zap.Uint("carrier_id", carrierID),
zap.Error(err))
}
return
}
if carrierType == constants.AssetTypeDevice {
bindings, err := h.deviceSimBinding.ListByDeviceID(ctx, carrierID)
if err != nil {
h.logger.Warn("套餐失效前查询设备绑定卡失败",
zap.Uint("device_id", carrierID),
zap.Error(err))
return
}
for _, b := range bindings {
if err := h.trafficSyncer.RefreshCardDataByID(ctx, b.IotCardID); err != nil {
h.logger.Warn("套餐失效前流量同步失败(设备绑定卡)",
zap.Uint("device_id", carrierID),
zap.Uint("card_id", b.IotCardID),
zap.Error(err))
}
}
}
}
// HandlePackageActivationCheck 任务 19.2-19.5: 处理套餐激活检查
// 每 10 秒调度一次,检查过期主套餐并激活下一个待生效主套餐
func (h *PackageActivationHandler) HandlePackageActivationCheck(ctx context.Context) error {
@@ -145,20 +194,21 @@ func (h *PackageActivationHandler) findAndActivateOrphanPackages(ctx context.Con
continue
}
// 检查该载体是否已有生效套餐
// 检查该载体是否已有占位主套餐(生效中或已用完均视为占位,不允许激活待生效套餐
var activeCount int64
var countErr error
occupiedStatuses := []int{constants.PackageUsageStatusActive, constants.PackageUsageStatusDepleted}
if key.carrierType == "iot_card" {
countErr = h.db.WithContext(ctx).
Model(&model.PackageUsage{}).
Where("status = ?", constants.PackageUsageStatusActive).
Where("status IN ?", occupiedStatuses).
Where("master_usage_id IS NULL").
Where("iot_card_id = ?", key.carrierID).
Count(&activeCount).Error
} else {
countErr = h.db.WithContext(ctx).
Model(&model.PackageUsage{}).
Where("status = ?", constants.PackageUsageStatusActive).
Where("status IN ?", occupiedStatuses).
Where("master_usage_id IS NULL").
Where("device_id = ?", key.carrierID).
Count(&activeCount).Error
@@ -171,7 +221,7 @@ func (h *PackageActivationHandler) findAndActivateOrphanPackages(ctx context.Con
continue
}
// 已有生效套餐,跳过
// 已有生效或已用完(占位)套餐,跳过
if activeCount > 0 {
continue
}
@@ -219,10 +269,15 @@ func (h *PackageActivationHandler) findExpiredMainPackages(ctx context.Context)
}
// processExpiredPackage 处理单个过期套餐
// triggerStopAfterExpiry 在事务提交后才调用,避免 goroutine 在 TX 提交前读到脏数据
// 流程:先同步最新流量 → 事务内标记过期/失效/激活下一个 → 事务提交后触发停机
func (h *PackageActivationHandler) processExpiredPackage(ctx context.Context, pkg *model.PackageUsage) error {
carrierType, carrierID := h.getCarrierInfo(pkg)
// 失效前先从 Gateway 同步一次最新流量,确保 data_usage_mb 精确
if carrierType != "" && carrierID > 0 {
h.syncTrafficBeforeExpiry(ctx, carrierType, carrierID)
}
err := h.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// 任务 19.3: 更新过期主套餐状态为 Expired (status=3)
if err := tx.Model(pkg).Update("status", constants.PackageUsageStatusExpired).Error; err != nil {

View File

@@ -52,4 +52,12 @@ func registerAgentRechargeRoutes(router fiber.Router, handler *admin.AgentRechar
Output: new(dto.AgentRechargeResponse),
Auth: true,
})
Register(group, doc, groupPath, "POST", "/:id/reject", handler.Reject, RouteSpec{
Summary: "驳回代理充值订单",
Tags: []string{"代理预充值"},
Input: new(dto.AgentRechargeRejectParams),
Output: nil,
Auth: true,
})
}

View File

@@ -320,15 +320,18 @@ func (s *Service) CreateWalletPackageOrders(ctx context.Context, req *dto.AgentO
resp.FailedCards = append(resp.FailedCards, buildFailedCard(rawCardNo, errors.New(errors.CodeInvalidParam, "卡标识不能为空")))
continue
}
card, err := s.resolveOpenAPICard(ctx, cardNo)
if err != nil {
resp.FailedCards = append(resp.FailedCards, buildFailedCard(cardNo, err))
continue
}
// 卡已绑定设备时,自动转为设备维度购买
identifier := card.ICCID
if !card.IsStandalone {
var identifier string
card, cardErr := s.resolveOpenAPICard(ctx, cardNo)
if cardErr != nil {
// 兜底:尝试解析为设备标识(支持 IMEI/虚拟号)
device, devErr := s.resolveOpenAPIDevice(ctx, cardNo)
if devErr != nil {
resp.FailedCards = append(resp.FailedCards, buildFailedCard(cardNo, cardErr))
continue
}
identifier = device.VirtualNo
} else if !card.IsStandalone {
// 卡已绑定设备时,自动转为设备维度购买
binding, bindErr := s.deviceSimBindingStore.GetActiveBindingByCardID(ctx, card.ID)
if bindErr != nil {
resp.FailedCards = append(resp.FailedCards, buildFailedCard(cardNo, errors.Wrap(errors.CodeInternalError, bindErr, "查询卡绑定设备失败")))
@@ -340,6 +343,8 @@ func (s *Service) CreateWalletPackageOrders(ctx context.Context, req *dto.AgentO
continue
}
identifier = device.VirtualNo
} else {
identifier = card.ICCID
}
orderResp, err := s.orderService.CreateAdminOrder(ctx, &dto.CreateAdminOrderRequest{
@@ -479,7 +484,7 @@ func (s *Service) resolveOpenAPICard(ctx context.Context, cardNo string) (*model
return nil, errors.New(errors.CodeInternalError, "开放接口资产解析依赖未配置")
}
resolved, err := s.assetService.Resolve(ctx, identifier)
resolved, err := s.assetService.Resolve(ctx, identifier, false)
if err != nil {
if appErr, ok := err.(*errors.AppError); ok && appErr.Code == errors.CodeNotFound {
return s.resolveOpenAPICardByIdentifier(ctx, identifier)

View File

@@ -387,6 +387,35 @@ func (s *Service) HandlePaymentCallback(ctx context.Context, rechargeNo string,
return nil
}
// Reject 驳回代理充值订单
// 仅 status=1待支付的订单可驳回驳回后状态变为 6已驳回为终态
func (s *Service) Reject(ctx context.Context, id uint, rejectionReason string) error {
record, err := s.agentRechargeStore.GetByID(ctx, id)
if err != nil {
if err == gorm.ErrRecordNotFound {
return errors.New(errors.CodeNotFound, "充值记录不存在")
}
return errors.Wrap(errors.CodeDatabaseError, err, "查询充值记录失败")
}
if record.Status != constants.RechargeStatusPending {
return errors.New(errors.CodeInvalidStatus, "仅待支付订单可驳回")
}
if err := s.agentRechargeStore.UpdateStatusWithRejection(ctx, id, rejectionReason); err != nil {
if err == gorm.ErrRecordNotFound {
return errors.New(errors.CodeInvalidStatus, "仅待支付订单可驳回")
}
return errors.Wrap(errors.CodeDatabaseError, err, "驳回充值订单失败")
}
s.logger.Info("代理充值订单驳回成功",
zap.Uint("record_id", id),
zap.String("rejection_reason", rejectionReason),
)
return nil
}
// GetByID 根据ID查询充值订单详情
// GET /api/admin/agent-recharges/:id
func (s *Service) GetByID(ctx context.Context, id uint) (*dto.AgentRechargeResponse, error) {
@@ -490,6 +519,7 @@ func toResponse(record *model.AgentRechargeRecord, shopName string) *dto.AgentRe
Remark: record.Remark,
Status: record.Status,
StatusName: constants.GetRechargeStatusName(record.Status),
RejectionReason: record.RejectionReason,
CreatedAt: record.CreatedAt.Format("2006-01-02 15:04:05"),
UpdatedAt: record.UpdatedAt.Format("2006-01-02 15:04:05"),
}

View File

@@ -91,7 +91,8 @@ func New(
// Resolve 通过任意标识符解析资产
// 主路径:查注册表(精确匹配 ICCID 或 VirtualNo
// Fallback原有跨表 OR 查询(处理 IMEI/SN/MSISDN 等非注册标识符)
func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetResolveResponse, error) {
// includeUsageSummary 为 true 时额外填充当前世代流量汇总字段
func (s *Service) Resolve(ctx context.Context, identifier string, includeUsageSummary bool) (*dto.AssetResolveResponse, error) {
if s.assetIdentifierStore != nil {
regRecord, regErr := s.assetIdentifierStore.FindByIdentifier(ctx, identifier)
if regErr == nil && regRecord != nil {
@@ -102,6 +103,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildDeviceResolveResponse(ctx, device)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "device", device.ID)
}
}
return resp, buildErr
}
@@ -111,6 +115,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildCardResolveResponse(ctx, card)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "iot_card", card.ID)
}
}
return resp, buildErr
}
@@ -123,6 +130,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildDeviceResolveResponse(ctx, device)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "device", device.ID)
}
}
return resp, buildErr
}
@@ -135,6 +145,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildCardResolveResponse(ctx, &card)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "iot_card", card.ID)
}
}
return resp, buildErr
}
@@ -336,6 +349,19 @@ func (s *Service) fillPackageInfo(ctx context.Context, resp *dto.AssetResolveRes
resp.EnableVirtualData = usage.EnableVirtualDataSnapshot
}
// fillUsageSummary 填充当前世代流量汇总字段。
// 仅在 includeUsageSummary=true 时被调用,无套餐时返回 0.0(非 null
func (s *Service) fillUsageSummary(ctx context.Context, resp *dto.AssetResolveResponse, carrierType string, carrierID uint) {
usages, err := s.packageUsageStore.GetCurrentGenerationUsagesForSummary(ctx, carrierType, carrierID)
if err != nil {
logger.GetAppLogger().Error("查询流量汇总失败", zap.String("carrierType", carrierType), zap.Uint("carrierID", carrierID), zap.Error(err))
return
}
totalUsed, totalRemaining := computeUsageSummary(usages)
resp.TotalVirtualUsedMB = &totalUsed
resp.TotalVirtualRemainingMB = &totalRemaining
}
// fillShopName 填充店铺名称
func (s *Service) fillShopName(ctx context.Context, resp *dto.AssetResolveResponse) {
if resp.ShopID == nil || *resp.ShopID == 0 {
@@ -971,7 +997,7 @@ func (s *Service) GetOrders(ctx context.Context, identifier string, page, pageSi
pageSize = 100
}
asset, err := s.Resolve(ctx, identifier)
asset, err := s.Resolve(ctx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -0,0 +1,16 @@
package asset
import "github.com/break/junhong_cmp_fiber/internal/model"
// computeUsageSummary 汇总当前世代套餐的虚已用量和虚剩余量。
// 对每条记录调用 BuildTrafficMetrics() 获取虚流量字段,累加后计算剩余。
func computeUsageSummary(usages []*model.PackageUsage) (totalUsedMB float64, totalRemainingMB float64) {
var totalVirtualTotal float64
for _, u := range usages {
m := u.BuildTrafficMetrics()
totalUsedMB += m.VirtualUsedMB
totalVirtualTotal += float64(m.VirtualTotalMB)
}
totalRemainingMB = totalVirtualTotal - totalUsedMB
return
}

View File

@@ -0,0 +1,81 @@
package asset
import (
"testing"
"github.com/break/junhong_cmp_fiber/internal/model"
)
func TestComputeUsageSummary_EmptyList(t *testing.T) {
totalUsed, totalRemaining := computeUsageSummary(nil)
if totalUsed != 0.0 {
t.Errorf("期望 totalUsed=0.0,实际=%v", totalUsed)
}
if totalRemaining != 0.0 {
t.Errorf("期望 totalRemaining=0.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_SingleVirtualPackage(t *testing.T) {
// 启用虚流量:真实总量=100MB虚拟总量=200MB倍率=0.5
// 真实已用=40MB → 虚拟已用=min(40*0.5, 100)=20MB
// 虚拟剩余=200-20=180MB
usage := &model.PackageUsage{
DataLimitMB: 100,
DataUsageMB: 40,
VirtualTotalMBSnapshot: 200,
DisplayGainRatioSnapshot: 0.5,
EnableVirtualDataSnapshot: true,
}
totalUsed, totalRemaining := computeUsageSummary([]*model.PackageUsage{usage})
if totalUsed != 20.0 {
t.Errorf("期望 totalUsed=20.0,实际=%v", totalUsed)
}
if totalRemaining != 180.0 {
t.Errorf("期望 totalRemaining=180.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_SingleNonVirtualPackage(t *testing.T) {
// 未启用虚流量:退化为真实值
// 真实总量=100MB真实已用=40MB → 虚拟已用=40MB虚拟剩余=60MB
usage := &model.PackageUsage{
DataLimitMB: 100,
DataUsageMB: 40,
EnableVirtualDataSnapshot: false,
}
totalUsed, totalRemaining := computeUsageSummary([]*model.PackageUsage{usage})
if totalUsed != 40.0 {
t.Errorf("期望 totalUsed=40.0,实际=%v", totalUsed)
}
if totalRemaining != 60.0 {
t.Errorf("期望 totalRemaining=60.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_MultiplePackages(t *testing.T) {
// 套餐1启用虚流量VirtualUsed=20, VirtualTotal=200
// 套餐2未启用虚流量VirtualUsed=10, VirtualTotal=50
// 汇总totalUsed=30, totalRemaining=(200+50)-30=220
usages := []*model.PackageUsage{
{
DataLimitMB: 100,
DataUsageMB: 40,
VirtualTotalMBSnapshot: 200,
DisplayGainRatioSnapshot: 0.5,
EnableVirtualDataSnapshot: true,
},
{
DataLimitMB: 50,
DataUsageMB: 10,
EnableVirtualDataSnapshot: false,
},
}
totalUsed, totalRemaining := computeUsageSummary(usages)
if totalUsed != 30.0 {
t.Errorf("期望 totalUsed=30.0,实际=%v", totalUsed)
}
if totalRemaining != 220.0 {
t.Errorf("期望 totalRemaining=220.0,实际=%v", totalRemaining)
}
}

View File

@@ -128,7 +128,7 @@ func (s *Service) CreateOrder(ctx context.Context, customerID uint, req *dto.Cli
}
skipCtx := context.WithValue(ctx, constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier))
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier), false)
if err != nil {
return nil, err
}
@@ -229,6 +229,18 @@ func (s *Service) CreateOrder(ctx context.Context, customerID uint, req *dto.Cli
}()
if forceRecharge.NeedForceRecharge {
// 强充第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// switch assetInfo.AssetType {
// case "card":
// if req.PaymentMethod != model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "单卡资产强充只支持支付宝支付")
// }
// case constants.ResourceTypeDevice:
// if req.PaymentMethod == model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "设备资产强充只支持微信支付")
// }
// }
if req.PaymentMethod == model.PaymentMethodAlipay {
// 支付宝强充:不需要 app_type / OpenID
activeConfig, err := s.wechatConfigService.GetActiveConfig(skipCtx)
@@ -1060,7 +1072,7 @@ func (s *Service) createAlipayForceRechargeOrder(
func (s *Service) ListOrders(ctx context.Context, customerID uint, req *dto.ClientOrderListRequest) ([]dto.ClientOrderListItem, int64, error) {
skipCtx := context.WithValue(ctx, constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier))
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier), false)
if err != nil {
return nil, 0, err
}
@@ -1231,7 +1243,7 @@ func (s *Service) PayOrder(ctx context.Context, customerID uint, orderID uint, r
if err == nil && device != nil {
assetRealnamePolicy = device.RealnamePolicy
// 设备的 RealNameStatus 需要通过 Resolve 获取(从当前卡派生)
resolved, resolveErr := s.assetService.Resolve(skipCtx, device.VirtualNo)
resolved, resolveErr := s.assetService.Resolve(skipCtx, device.VirtualNo, false)
if resolveErr == nil && resolved != nil {
assetRealnameStatus = resolved.RealNameStatus
}
@@ -1241,6 +1253,20 @@ func (s *Service) PayOrder(ctx context.Context, customerID uint, orderID uint, r
return nil, errors.New(errors.CodeNeedRealname)
}
// 第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// if req.PaymentMethod == model.PaymentMethodWechat || req.PaymentMethod == model.PaymentMethodAlipay {
// switch order.OrderType {
// case model.OrderTypeSingleCard:
// if req.PaymentMethod != model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "单卡资产只支持支付宝支付")
// }
// case model.OrderTypeDevice:
// if req.PaymentMethod != model.PaymentMethodWechat {
// return nil, errors.New(errors.CodeInvalidParam, "设备资产只支持微信支付")
// }
// }
// }
switch req.PaymentMethod {
case model.PaymentMethodWallet:
if err := s.orderPaymentService.WalletPay(skipCtx, orderID, model.BuyerTypePersonal, customerID); err != nil {

View File

@@ -29,6 +29,8 @@ type PollingCallback interface {
OnCardCreated(ctx context.Context, card *model.IotCard)
// OnCardStatusChanged 卡状态变化时的回调
OnCardStatusChanged(ctx context.Context, cardID uint)
// OnBatchCardsStatusChanged 批量卡状态变化时的回调(批量分配/回收场景)
OnBatchCardsStatusChanged(ctx context.Context, cardIDs []uint)
// OnCardDeleted 卡删除时的回调
OnCardDeleted(ctx context.Context, cardID uint)
// OnCardEnabled 卡启用轮询时的回调
@@ -597,6 +599,7 @@ func (s *Service) AllocateCards(ctx context.Context, req *dto.AllocateStandalone
newStatus := constants.IotCardStatusDistributed
toShopID := req.ToShopID
allocationNo := s.assetAllocationRecordStore.GenerateAllocationNo(ctx, constants.AssetAllocationTypeAllocate)
err = s.db.Transaction(func(tx *gorm.DB) error {
txIotCardStore := postgres.NewIotCardStore(tx, nil)
@@ -606,7 +609,6 @@ func (s *Service) AllocateCards(ctx context.Context, req *dto.AllocateStandalone
return err
}
allocationNo := s.assetAllocationRecordStore.GenerateAllocationNo(ctx, constants.AssetAllocationTypeAllocate)
records := s.buildAllocationRecords(cards, cardIDs, operatorShopID, toShopID, operatorID, allocationNo, req.Remark)
return txRecordStore.BatchCreate(ctx, records)
})
@@ -631,11 +633,14 @@ func (s *Service) AllocateCards(ctx context.Context, req *dto.AllocateStandalone
}
s.iotCardStore.InvalidateListCountCache(ctx)
// 通知轮询调度器状态变化(卡被分配后可能需要重新匹配配置
// 通知轮询调度器状态变化(异步执行 + 批量操作,避免 N 次单卡 DB 查询打满连接池
if s.pollingCallback != nil && len(cardIDs) > 0 {
for _, cardID := range cardIDs {
s.pollingCallback.OnCardStatusChanged(ctx, cardID)
}
cardIDsCopy := make([]uint, len(cardIDs))
copy(cardIDsCopy, cardIDs)
cb := s.pollingCallback
go func() {
cb.OnBatchCardsStatusChanged(context.Background(), cardIDsCopy)
}()
}
shopMap := s.loadShopNames(ctx, cards)
@@ -672,7 +677,7 @@ func (s *Service) AllocateCards(ctx context.Context, req *dto.AllocateStandalone
TotalCount: len(cards),
SuccessCount: len(cardIDs),
FailCount: len(failedItems),
AllocationNo: s.assetAllocationRecordStore.GenerateAllocationNo(ctx, constants.AssetAllocationTypeAllocate),
AllocationNo: allocationNo,
FailedItems: failedItems,
}, nil
}
@@ -844,11 +849,14 @@ func (s *Service) RecallCards(ctx context.Context, req *dto.RecallStandaloneCard
}
s.iotCardStore.InvalidateListCountCache(ctx)
// 通知轮询调度器状态变化(卡被回收后可能需要重新匹配配置
// 通知轮询调度器状态变化(异步批量执行,避免回收大批卡时打满 DB 连接池
if s.pollingCallback != nil && len(cardIDs) > 0 {
for _, cardID := range cardIDs {
s.pollingCallback.OnCardStatusChanged(ctx, cardID)
}
cardIDsCopy := make([]uint, len(cardIDs))
copy(cardIDsCopy, cardIDs)
cb := s.pollingCallback
go func() {
cb.OnBatchCardsStatusChanged(context.Background(), cardIDsCopy)
}()
}
shopMap := s.loadShopNames(ctx, successCards)
@@ -1486,6 +1494,19 @@ func (s *Service) buildCardNotFoundFailedItems(iccids []string) []dto.CardSeries
return items
}
// RefreshCardDataByID 通过卡 ID 从 Gateway 同步卡数据
// 内部查询 ICCID 后委托给 RefreshCardDataFromGateway供套餐失效前流量同步使用
func (s *Service) RefreshCardDataByID(ctx context.Context, cardID uint) error {
card, err := s.iotCardStore.GetByID(ctx, cardID)
if err != nil {
if err == gorm.ErrRecordNotFound {
return nil
}
return err
}
return s.RefreshCardDataFromGateway(ctx, card.ICCID)
}
// RefreshCardDataFromGateway 从 Gateway 完整同步卡数据
// 调用网关查询网络状态、实名状态、本月流量,并写回数据库
// 流量采用增量计算与轮询逻辑一致increment = 当前网关读数 - 上次网关读数

View File

@@ -1120,26 +1120,31 @@ func (s *Service) createOrderWithWalletPayment(ctx context.Context, order *model
}
actualAmount := *order.ActualPaidAmount
// 1. 事务外:检查钱包余额(快速失败)
wallet, err := s.agentWalletStore.GetMainWallet(ctx, operatorShopID)
if err != nil {
if err == gorm.ErrRecordNotFound {
return errors.New(errors.CodeWalletNotFound, "钱包不存在")
// 事务内:加锁读取钱包 + 余额检查 + 创建订单 + 扣款 + 创建流水 + 激活套餐
// 使用 FOR UPDATE 悲观锁替代乐观锁,避免并发请求因 version 过期导致误报余额不足
err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// 1. 加锁读取钱包,并发请求排队等待,不会冲突
var wallet model.AgentWallet
if err := tx.Set("gorm:query_option", "FOR UPDATE").
Where("shop_id = ? AND wallet_type = ?", operatorShopID, constants.AgentWalletTypeMain).
First(&wallet).Error; err != nil {
if err == gorm.ErrRecordNotFound {
return errors.New(errors.CodeWalletNotFound, "钱包不存在")
}
return errors.Wrap(errors.CodeDatabaseError, err, "查询钱包失败")
}
return errors.Wrap(errors.CodeDatabaseError, err, "查询钱包失败")
}
if wallet.Balance < actualAmount {
return errors.New(errors.CodeInsufficientBalance, "余额不足")
}
// 2. 事务内:创建订单 + 扣款 + 创建流水 + 激活套餐
err = s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// 2.1 创建订单
// 2. 余额检查(加锁后读到的是最新余额,结果准确)
if wallet.Balance < actualAmount {
return errors.New(errors.CodeInsufficientBalance, "余额不足")
}
// 3. 创建订单
if err := tx.Create(order).Error; err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "创建订单失败")
}
// 2.2 创建订单明细
// 4. 创建订单明细
for _, item := range items {
item.OrderID = order.ID
}
@@ -1147,30 +1152,24 @@ func (s *Service) createOrderWithWalletPayment(ctx context.Context, order *model
return errors.Wrap(errors.CodeDatabaseError, err, "创建订单明细失败")
}
// 2.3 扣减钱包余额(乐观锁
result := tx.Model(&model.AgentWallet{}).
Where("id = ? AND balance >= ? AND version = ?", wallet.ID, actualAmount, wallet.Version).
Updates(map[string]any{
"balance": gorm.Expr("balance - ?", actualAmount),
"version": gorm.Expr("version + 1"),
})
if result.Error != nil {
return errors.Wrap(errors.CodeDatabaseError, result.Error, "扣减钱包余额失败")
}
if result.RowsAffected == 0 {
return errors.New(errors.CodeInsufficientBalance, "余额不足或并发冲突")
// 5. 扣减钱包余额(FOR UPDATE 已保证串行,无需 version 条件
if err := tx.Model(&wallet).Updates(map[string]any{
"balance": gorm.Expr("balance - ?", actualAmount),
"version": gorm.Expr("version + 1"),
}).Error; err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "扣减钱包余额失败")
}
// 2.4 创建钱包流水
// 6. 创建钱包流水
var relatedShopID *uint
if order.PurchaseRole == model.PurchaseRolePurchaseForSubordinate {
relatedShopID = &buyerShopID
}
if err := s.createWalletTransaction(ctx, tx, wallet, order, actualAmount, order.PurchaseRole, relatedShopID); err != nil {
if err := s.createWalletTransaction(ctx, tx, &wallet, order, actualAmount, order.PurchaseRole, relatedShopID); err != nil {
return err
}
// 2.5 激活套餐
// 7. 激活套餐
if err := s.activatePackage(ctx, tx, order); err != nil {
return err
}

View File

@@ -92,19 +92,20 @@ func (s *ActivationService) ActivateByRealname(ctx context.Context, carrierType
// 检查是否是主套餐
if usage.MasterUsageID == nil {
// 主套餐:需要检查是否有已激活的主套餐
// 主套餐:需要检查是否有已激活或已用完(仍在有效期)的主套餐
var activeMain model.PackageUsage
err := tx.Where("status = ?", constants.PackageUsageStatusActive).
err := tx.Where("status IN ?", []int{constants.PackageUsageStatusActive, constants.PackageUsageStatusDepleted}).
Where("master_usage_id IS NULL").
Where(carrierType+"_id = ?", carrierID).
Order("priority ASC").
First(&activeMain).Error
if err == nil {
// 已有激活的主套餐,保持排队状态
s.logger.Warn("已有激活主套餐,跳过激活",
// 已有生效中或已用完(占位)的主套餐,保持排队状态
s.logger.Warn("已有占位主套餐,跳过激活",
zap.Uint("usage_id", usage.ID),
zap.Uint("active_main_id", activeMain.ID))
zap.Uint("active_main_id", activeMain.ID),
zap.Int("active_main_status", activeMain.Status))
continue
}
if err != gorm.ErrRecordNotFound {
@@ -500,9 +501,10 @@ func (s *ActivationService) getNextPendingMainPackage(ctx context.Context, tx *g
func (s *ActivationService) hasActiveMainPackage(ctx context.Context, tx *gorm.DB, carrierType string, carrierID uint) (bool, error) {
var count int64
// 生效中1和已用完2都属于"占位"状态:已用完的套餐仍在有效期内,不允许提前激活下一个主套餐
if err := tx.WithContext(ctx).
Model(&model.PackageUsage{}).
Where("status = ?", constants.PackageUsageStatusActive).
Where("status IN ?", []int{constants.PackageUsageStatusActive, constants.PackageUsageStatusDepleted}).
Where("master_usage_id IS NULL").
Where(carrierType+"_id = ?", carrierID).
Count(&count).Error; err != nil {

View File

@@ -88,13 +88,22 @@ func (s *UsageService) DeductDataUsage(ctx context.Context, carrierType string,
return err
}
today := time.Now().Format("2006-01-02")
if len(packages) == 0 {
// 无生效套餐:尝试找 Depleted 套餐记录溢出流量详单,避免详单断档
if recordErr := s.recordOverflowToDepletedPackage(ctx, tx, targetCarrierType, targetCarrierID, deductUsageMB, today); recordErr != nil {
s.logger.Warn("记录溢出流量到已耗尽套餐失败",
zap.String("carrier_type", targetCarrierType),
zap.Uint("carrier_id", targetCarrierID),
zap.Int64("usage_mb", deductUsageMB),
zap.Error(recordErr))
}
return errors.New(errors.CodeNoAvailablePackage, "没有可用套餐")
}
// 按优先级扣减流量
remainingUsage := deductUsageMB
today := time.Now().Format("2006-01-02")
for index, pkg := range packages {
if remainingUsage <= 0 {
@@ -446,3 +455,45 @@ func (s *UsageService) triggerSuspensionAfterCommit(carrierType string, carrierI
}(cardID)
}
}
// recordOverflowToDepletedPackage 在所有套餐已耗尽时,将溢出流量写入最近的 Depleted 套餐日记录。
// 目的:保持流量详单连续性,即便套餐用完期间上游仍有流量使用,详单不会断档。
// 只写日记录和累加 data_usage_mb不改变套餐 status不触发停机。
func (s *UsageService) recordOverflowToDepletedPackage(ctx context.Context, tx *gorm.DB, carrierType string, carrierID uint, overflowMB int64, today string) error {
query := tx.Where("status = ?", constants.PackageUsageStatusDepleted)
switch carrierType {
case constants.AssetTypeIotCard:
query = query.Where("iot_card_id = ?", carrierID)
case constants.AssetTypeDevice:
query = query.Where("device_id = ?", carrierID)
default:
return nil
}
// 优先选主套餐master_usage_id IS NULL主套餐不存在时再退而求其次取加油包
var pkg model.PackageUsage
err := query.Where("master_usage_id IS NULL").Order("id DESC").First(&pkg).Error
if err == gorm.ErrRecordNotFound {
err = query.Order("id DESC").First(&pkg).Error
}
if err != nil {
// 找不到任何 Depleted 套餐,静默忽略
return nil
}
newUsage := pkg.DataUsageMB + overflowMB
if err := tx.Model(&pkg).Update("data_usage_mb", newUsage).Error; err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "更新溢出流量失败")
}
if err := s.updateDailyRecord(ctx, tx, pkg.ID, today, overflowMB, newUsage); err != nil {
return err
}
s.logger.Info("记录溢出流量到已耗尽套餐",
zap.Uint("package_usage_id", pkg.ID),
zap.Int64("overflow_mb", overflowMB),
zap.Int64("new_usage_mb", newUsage))
return nil
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/redis/go-redis/v9"
"gorm.io/gorm"
)
@@ -74,6 +75,25 @@ func (s *AgentRechargeStore) Update(ctx context.Context, record *model.AgentRech
return s.db.WithContext(ctx).Save(record).Error
}
// UpdateStatusWithRejection 条件更新状态为已驳回并写入驳回原因
// 仅当 status = 1待支付时更新RowsAffected=0 说明订单不存在或状态已变更
func (s *AgentRechargeStore) UpdateStatusWithRejection(ctx context.Context, id uint, rejectionReason string) error {
result := s.db.WithContext(ctx).
Model(&model.AgentRechargeRecord{}).
Where("id = ? AND status = ?", id, constants.RechargeStatusPending).
Updates(map[string]interface{}{
"status": constants.RechargeStatusRejected,
"rejection_reason": rejectionReason,
})
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return nil
}
// UpdateWithTx 更新充值记录(带事务)
func (s *AgentRechargeStore) UpdateWithTx(ctx context.Context, tx *gorm.DB, record *model.AgentRechargeRecord) error {
return tx.WithContext(ctx).Save(record).Error

View File

@@ -67,6 +67,16 @@ func (s *DeviceImportTaskStore) UpdateStatus(ctx context.Context, id uint, statu
return s.db.WithContext(ctx).Model(&model.DeviceImportTask{}).Where("id = ?", id).Updates(updates).Error
}
// UpdateProgress 实时更新导入进度计数(不更新详情列表,用于批量处理中间状态)
func (s *DeviceImportTaskStore) UpdateProgress(ctx context.Context, id uint, successCount, skipCount, failCount int) error {
return s.db.WithContext(ctx).Model(&model.DeviceImportTask{}).Where("id = ?", id).Updates(map[string]any{
"success_count": successCount,
"skip_count": skipCount,
"fail_count": failCount,
"updated_at": time.Now(),
}).Error
}
func (s *DeviceImportTaskStore) UpdateResult(ctx context.Context, id uint, totalCount, successCount, skipCount, failCount, warningCount int, skippedItems, failedItems, warningItems model.ImportResultItems) error {
updates := map[string]any{
"total_count": totalCount,

View File

@@ -72,6 +72,16 @@ func (s *IotCardImportTaskStore) UpdateStatus(ctx context.Context, id uint, stat
return s.db.WithContext(ctx).Model(&model.IotCardImportTask{}).Where("id = ?", id).Updates(updates).Error
}
// UpdateProgress 实时更新导入进度计数(不更新详情列表,用于批量处理中间状态)
func (s *IotCardImportTaskStore) UpdateProgress(ctx context.Context, id uint, successCount, skipCount, failCount int) error {
return s.db.WithContext(ctx).Model(&model.IotCardImportTask{}).Where("id = ?", id).Updates(map[string]interface{}{
"success_count": successCount,
"skip_count": skipCount,
"fail_count": failCount,
"updated_at": time.Now(),
}).Error
}
func (s *IotCardImportTaskStore) UpdateResult(ctx context.Context, id uint, successCount, skipCount, failCount int, skippedItems, failedItems model.ImportResultItems) error {
updates := map[string]interface{}{
"success_count": successCount,

View File

@@ -301,6 +301,40 @@ func (s *PackageUsageStore) ListByCarrier(ctx context.Context, carrierType strin
return usages, nil
}
// GetCurrentGenerationUsagesForSummary 查询当前世代内所有已激活过的套餐记录(含加油包),用于流量汇总。
// 当前世代定义status∈{1,2,3,4} 中 generation 最大值所在的那一批记录。
// 返回计算所需字段DataUsageMB、DataLimitMB、VirtualTotalMBSnapshot、DisplayGainRatioSnapshot、EnableVirtualDataSnapshot。
func (s *PackageUsageStore) GetCurrentGenerationUsagesForSummary(ctx context.Context, carrierType string, carrierID uint) ([]*model.PackageUsage, error) {
var usages []*model.PackageUsage
// 子查询取当前世代编号
subQuery := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Select("MAX(generation)").
Where("status IN ?", []int{
constants.PackageUsageStatusActive,
constants.PackageUsageStatusDepleted,
constants.PackageUsageStatusExpired,
constants.PackageUsageStatusInvalidated,
})
subQuery = applyPackageUsageCarrierFilter(subQuery, carrierType, carrierID)
query := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Select("data_usage_mb, data_limit_mb, virtual_total_mb_snapshot, display_gain_ratio_snapshot, enable_virtual_data_snapshot").
Where("status IN ?", []int{
constants.PackageUsageStatusActive,
constants.PackageUsageStatusDepleted,
constants.PackageUsageStatusExpired,
constants.PackageUsageStatusInvalidated,
}).
Where("generation = (?)", subQuery)
query = applyPackageUsageCarrierFilter(query, carrierType, carrierID)
if err := query.Find(&usages).Error; err != nil {
return nil, err
}
return usages, nil
}
func applyPackageUsageCarrierFilter(query *gorm.DB, carrierType string, carrierID uint) *gorm.DB {
if carrierType == "iot_card" {
return query.Where("iot_card_id = ?", carrierID)

View File

@@ -85,7 +85,18 @@ func (h *DeviceImportHandler) HandleDeviceImport(ctx context.Context, task *asyn
return asynq.SkipRetry
}
if importTask.Status != model.ImportTaskStatusPending {
switch importTask.Status {
case model.ImportTaskStatusPending:
// 正常首次处理
case model.ImportTaskStatusProcessing:
// 上次 worker 中途中断(重启/崩溃),重置计数重新处理
// 已入库的设备会被 ExistsByDeviceNoBatch 识别为 skip不会重复写入
h.logger.Warn("检测到导入任务上次处理中途中断,重置进度重新处理",
zap.Uint("task_id", payload.TaskID),
)
h.importTaskStore.UpdateProgress(ctx, importTask.ID, 0, 0, 0)
default:
// 已完成或失败,不重复处理
h.logger.Info("导入任务已处理,跳过",
zap.Uint("task_id", payload.TaskID),
zap.Int("status", importTask.Status),
@@ -185,6 +196,8 @@ func (h *DeviceImportHandler) processImport(ctx context.Context, task *model.Dev
end := min(i+deviceBatchSize, len(rows))
batch := rows[i:end]
h.processBatch(ctx, task, batch, result)
// 每批完成后实时更新进度计数,让前端可以看到处理进度
h.importTaskStore.UpdateProgress(ctx, task.ID, result.successCount, result.skipCount, result.failCount)
}
return result

View File

@@ -41,15 +41,14 @@ type PollingCallback interface {
}
type IotCardImportHandler struct {
db *gorm.DB
redis *redis.Client
importTaskStore *postgres.IotCardImportTaskStore
iotCardStore *postgres.IotCardStore
assetWalletStore *postgres.AssetWalletStore
assetIdentifierStore *postgres.AssetIdentifierStore
storageService *storage.Service
pollingCallback PollingCallback
logger *zap.Logger
db *gorm.DB
redis *redis.Client
importTaskStore *postgres.IotCardImportTaskStore
iotCardStore *postgres.IotCardStore
assetWalletStore *postgres.AssetWalletStore
storageService *storage.Service
pollingCallback PollingCallback
logger *zap.Logger
}
func NewIotCardImportHandler(
@@ -58,21 +57,19 @@ func NewIotCardImportHandler(
importTaskStore *postgres.IotCardImportTaskStore,
iotCardStore *postgres.IotCardStore,
assetWalletStore *postgres.AssetWalletStore,
assetIdentifierStore *postgres.AssetIdentifierStore,
storageSvc *storage.Service,
pollingCallback PollingCallback,
logger *zap.Logger,
) *IotCardImportHandler {
return &IotCardImportHandler{
db: db,
redis: redis,
importTaskStore: importTaskStore,
iotCardStore: iotCardStore,
assetWalletStore: assetWalletStore,
assetIdentifierStore: assetIdentifierStore,
storageService: storageSvc,
pollingCallback: pollingCallback,
logger: logger,
db: db,
redis: redis,
importTaskStore: importTaskStore,
iotCardStore: iotCardStore,
assetWalletStore: assetWalletStore,
storageService: storageSvc,
pollingCallback: pollingCallback,
logger: logger,
}
}
@@ -95,7 +92,18 @@ func (h *IotCardImportHandler) HandleIotCardImport(ctx context.Context, task *as
return asynq.SkipRetry
}
if importTask.Status != model.ImportTaskStatusPending {
switch importTask.Status {
case model.ImportTaskStatusPending:
// 正常首次处理
case model.ImportTaskStatusProcessing:
// 上次 worker 中途中断(重启/崩溃),重置计数重新处理
// 已入库的卡会被 ExistsByICCIDBatch 识别为 skip不会重复写入
h.logger.Warn("检测到导入任务上次处理中途中断,重置进度重新处理",
zap.Uint("task_id", payload.TaskID),
)
h.importTaskStore.UpdateProgress(ctx, importTask.ID, 0, 0, 0)
default:
// 已完成或失败,不重复处理
h.logger.Info("导入任务已处理,跳过",
zap.Uint("task_id", payload.TaskID),
zap.Int("status", importTask.Status),
@@ -218,6 +226,8 @@ func (h *IotCardImportHandler) processImport(ctx context.Context, task *model.Io
end := min(i+batchSize, len(cards))
batch := cards[i:end]
h.processBatch(ctx, task, batch, i+1, result)
// 每批完成后实时更新进度计数,让前端可以看到处理进度
h.importTaskStore.UpdateProgress(ctx, task.ID, result.successCount, result.skipCount, result.failCount)
}
return result
@@ -341,7 +351,14 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
}
now := time.Now()
createdCards := make([]*model.IotCard, 0, len(finalCards))
// 构建待批量写入的卡列表,同时记录原始信息用于失败回溯
type pendingEntry struct {
item model.CardItem
meta cardMeta
}
pending := make([]pendingEntry, 0, len(finalCards))
iotCards := make([]*model.IotCard, 0, len(finalCards))
for _, card := range finalCards {
meta := cardMetaMap[card.ICCID]
@@ -386,43 +403,63 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
iotCard.CreatedAt = now
iotCard.UpdatedAt = now
txErr := h.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Create(iotCard).Error; err != nil {
return err
}
txIdentifierStore := postgres.NewAssetIdentifierStore(tx)
if err := txIdentifierStore.Register(ctx, card.ICCID, model.AssetTypeIotCard, iotCard.ID); err != nil {
return fmt.Errorf("ICCID 已被占用: %s", card.ICCID)
}
if card.VirtualNo != "" {
if err := txIdentifierStore.Register(ctx, card.VirtualNo, model.AssetTypeIotCard, iotCard.ID); err != nil {
return fmt.Errorf("虚拟号已被占用: %s", card.VirtualNo)
}
}
return nil
})
if txErr != nil {
h.logger.Error("创建 IoT 卡失败",
zap.String("iccid", card.ICCID),
zap.Error(txErr),
)
result.failedItems = append(result.failedItems, model.ImportResultItem{
Line: meta.line,
ICCID: card.ICCID,
MSISDN: meta.msisdn,
Reason: txErr.Error(),
})
result.failCount++
continue
}
createdCards = append(createdCards, iotCard)
result.successCount++
iotCards = append(iotCards, iotCard)
pending = append(pending, pendingEntry{item: card, meta: meta})
}
h.batchCreateWallets(ctx, createdCards)
if len(iotCards) == 0 {
return
}
// 整批一个事务:批量写入 iot_card + asset_identifier
// 相比逐卡事务,将 N 个事务压缩为 1 个,大幅减少数据库往返
txErr := h.db.Transaction(func(tx *gorm.DB) error {
// 批量插入 iot_cardGORM 会通过 RETURNING 回填所有 ID
if err := tx.CreateInBatches(&iotCards, 500).Error; err != nil {
return err
}
// 用回填的 ID 构建 asset_identifier 列表
identifiers := make([]*model.AssetIdentifier, 0, len(iotCards)*2)
for _, c := range iotCards {
identifiers = append(identifiers, &model.AssetIdentifier{
Identifier: c.ICCID,
AssetType: model.AssetTypeIotCard,
AssetID: c.ID,
})
if c.VirtualNo != "" {
identifiers = append(identifiers, &model.AssetIdentifier{
Identifier: c.VirtualNo,
AssetType: model.AssetTypeIotCard,
AssetID: c.ID,
})
}
}
return tx.CreateInBatches(&identifiers, 500).Error
})
if txErr != nil {
h.logger.Error("批量创建 IoT 卡失败",
zap.Error(txErr),
zap.Int("batch_size", len(iotCards)),
)
for _, p := range pending {
result.failedItems = append(result.failedItems, model.ImportResultItem{
Line: p.meta.line,
ICCID: p.item.ICCID,
MSISDN: p.meta.msisdn,
Reason: "批量写入失败: " + txErr.Error(),
})
result.failCount++
}
return
}
result.successCount += len(iotCards)
h.batchCreateWallets(ctx, iotCards)
if h.pollingCallback != nil {
h.pollingCallback.OnBatchCardsCreated(ctx, createdCards)
h.pollingCallback.OnBatchCardsCreated(ctx, iotCards)
}
}

View File

@@ -0,0 +1,2 @@
ALTER TABLE tb_agent_recharge_record
DROP COLUMN IF EXISTS rejection_reason;

View File

@@ -0,0 +1,5 @@
-- 为代理预充值表新增驳回原因字段,支持管理员驳回待支付订单
ALTER TABLE tb_agent_recharge_record
ADD COLUMN IF NOT EXISTS rejection_reason VARCHAR(500) NULL;
COMMENT ON COLUMN tb_agent_recharge_record.rejection_reason IS '驳回原因,仅驳回时写入';

View File

@@ -0,0 +1,6 @@
-- 回滚将状态约束恢复为不包含驳回状态6
ALTER TABLE tb_agent_recharge_record
DROP CONSTRAINT IF EXISTS chk_agent_recharge_status;
ALTER TABLE tb_agent_recharge_record
ADD CONSTRAINT chk_agent_recharge_status CHECK (status IN (1, 2, 3, 4, 5));

View File

@@ -0,0 +1,6 @@
-- 更新代理预充值状态检查约束新增驳回状态6
ALTER TABLE tb_agent_recharge_record
DROP CONSTRAINT IF EXISTS chk_agent_recharge_status;
ALTER TABLE tb_agent_recharge_record
ADD CONSTRAINT chk_agent_recharge_status CHECK (status IN (1, 2, 3, 4, 5, 6));

View File

@@ -98,6 +98,7 @@ const (
RechargeStatusCompleted = 3 // 已完成
RechargeStatusClosed = 4 // 已关闭
RechargeStatusRefunded = 5 // 已退款
RechargeStatusRejected = 6 // 已驳回
)
// 充值支付方式
@@ -121,7 +122,7 @@ const (
// ========== Redis Key 生成函数 ==========
// GetRechargeStatusName 获取充值状态名称
// 对应 RechargeStatus* 常量1=待支付, 2=已支付, 3=已完成, 4=已关闭, 5=已退款
// 对应 RechargeStatus* 常量1=待支付, 2=已支付, 3=已完成, 4=已关闭, 5=已退款, 6=已驳回
func GetRechargeStatusName(status int) string {
switch status {
case RechargeStatusPending:
@@ -134,6 +135,8 @@ func GetRechargeStatusName(status int) string {
return "已关闭"
case RechargeStatusRefunded:
return "已退款"
case RechargeStatusRejected:
return "已驳回"
default:
return "未知"
}

View File

@@ -88,7 +88,6 @@ func (h *Handler) registerIotCardImportHandler() {
h.workerResult.Stores.IotCardImportTask,
h.workerResult.Stores.IotCard,
h.workerResult.Stores.AssetWallet,
h.workerResult.Stores.AssetIdentifier,
h.storage,
h.pollingCallback,
h.logger,

View File

@@ -6,12 +6,29 @@
from __future__ import annotations
import contextlib
import time
from dataclasses import dataclass
from decimal import Decimal
from typing import Any, Iterable, Iterator, Optional
BATCH_SIZE = 200
# 重构后分阶段索引查询已无大 JOIN,1000 为安全默认值
BATCH_SIZE = 1000
class _StageTimer:
"""阶段耗时计时器,记录迁移各阶段用时。"""
def __init__(self, func_name: str) -> None:
self._func = func_name
self._t = time.perf_counter()
def log(self, stage: str, **kw: Any) -> None:
elapsed = time.perf_counter() - self._t
parts = " ".join(f"{k}={v}" for k, v in kw.items())
suffix = f" {parts}" if parts else ""
print(f"[legacy] {self._func}.{stage}:{suffix} elapsed={elapsed:.1f}s")
self._t = time.perf_counter()
def _normalize_iccid_pairs(iccid_pairs: Iterable[tuple]) -> list[tuple[str, str, bool]]:
@@ -236,14 +253,35 @@ def _to_mb_decimal(value) -> Decimal:
return Decimal("0")
def _build_card_lookup(
pairs: list[tuple[str, str, bool]],
) -> dict[str, tuple[str, int]]:
"""构建 tbl_card 查询键 → (完整 ICCID, rank) 映射。
rank=0 表示完整精确命中,rank=1 表示 19 位兜底命中。
"""
by_full: dict[str, tuple[str, int]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
by_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
by_full[i19] = (card_key, 1)
return by_full
# ---------------- 查询函数 ----------------
def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPackage]:
def fetch_current_packages(
conn,
iccids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyPackage]:
"""查每张卡的当前生效主套餐:status=1、type!=1 且 expire_date 最大。
返回: iccid → LegacyPackage(找不到的 iccid 不在结果中)
"""
timer = _StageTimer("fetch_current_packages")
iccid_list = sorted({i for i in iccids if i})
out: dict[str, LegacyPackage] = {}
if not iccid_list:
@@ -266,11 +304,15 @@ def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPacka
ORDER BY iccid_mark ASC, expire_date DESC
"""
seen: set[str] = set()
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
rows += 1
if not iccid or iccid in seen:
continue
seen.add(iccid)
@@ -284,15 +326,21 @@ def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPacka
flow_size_mb=int(row.get("flow_size") or 0),
status=int(row.get("status") or 0),
)
timer.log("tbl_card_life", keys=len(iccid_list), batches=batches, rows=rows, found=len(out))
return out
def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, list[LegacyPackageLifecycle]]:
def fetch_package_lifecycles(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> dict[str, list[LegacyPackageLifecycle]]:
"""查当前生效和未生效待生效正式套餐生命周期。
只返回可迁移范围(active/pending)和需要在审核文件中说明的跳过记录。
加油包(type=1)和已过期记录不生成套餐 SQL,但保留 skip_reason 供审核产物输出。
"""
timer = _StageTimer("fetch_package_lifecycles")
pairs = _normalize_iccid_pairs(iccid_pairs)
out: dict[str, list[LegacyPackageLifecycle]] = {}
if not pairs:
@@ -328,8 +376,11 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
COALESCE(expire_date, '9999-12-31') ASC,
id ASC
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row.get("iccid_mark") or ""
@@ -338,6 +389,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
iccid = query_to_full.get(legacy_iccid)
if not iccid:
continue
rows += 1
start = row.get("start_date")
expire = row.get("expire_date")
start_str = start.strftime("%Y-%m-%d %H:%M:%S") if start else None
@@ -363,6 +415,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
skip_reason=skip_reason,
)
out.setdefault(iccid, []).append(item)
timer.log("tbl_card_life", keys=len(iccid_list), batches=batches, rows=rows)
next_sql = """
SELECT
@@ -384,8 +437,11 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
COALESCE(effect_complete_time, '9999-12-31') ASC,
id ASC
"""
next_rows = 0
next_batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
next_batches += 1
cur.execute(next_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row.get("iccid_mark") or ""
@@ -394,6 +450,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
iccid = query_to_full.get(legacy_iccid)
if not iccid:
continue
next_rows += 1
start = row.get("start_date")
expire = row.get("expire_date")
start_str = start.strftime("%Y-%m-%d %H:%M:%S") if start else None
@@ -417,6 +474,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
migration_status=migration_status,
skip_reason=skip_reason,
))
timer.log("tbl_next_month_card_life", keys=len(iccid_list), batches=next_batches, rows=next_rows)
return out
@@ -448,13 +506,18 @@ def _classify_next_month_lifecycle(status: int) -> tuple[str, str]:
return "skipped", f"奇成次月待生效状态 {status} 不在迁移范围"
def fetch_card_usage(conn, iccid_pairs: Iterable[tuple]) -> dict[str, LegacyCardUsage]:
"""查每张卡的累计已用流量,并同步取当前生效正式套餐的 flow_add_discount。
def fetch_card_usage(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyCardUsage]:
"""查每张卡的累计已用流量,分阶段索引查询替代大 JOIN。
奇成 ``tbl_card.total_bytes_cnt`` 是虚用量(已被套餐虚比加成)。
这里 JOIN 当前生效正式套餐(``tbl_card_life`` status=1,type!=1 且 ``expire_date`` 最大)
+ ``tbl_set_meal`` 取 ``flow_add_discount``,放进 ``LegacyCardUsage``;
上层通过 ``real_used_mb_floor`` 反算真用量写入新系统。
阶段:
1. tbl_card — iccid_mark + total_bytes_cnt
2. tbl_card_life — 当前生效正式套餐 meal_id(Python 选 expire_date 最大)
3. tbl_set_meal — flow_add_discount(按 meal_id 精确查)
4. Python 组装 LegacyCardUsage
Args:
iccid_pairs: (iccid_19, iccid_20, allow_iccid_19_lookup) 列表。
@@ -462,88 +525,119 @@ def fetch_card_usage(conn, iccid_pairs: Iterable[tuple]) -> dict[str, LegacyCard
Returns:
{完整 ICCID: LegacyCardUsage}。
"""
timer = _StageTimer("fetch_card_usage")
pairs = _normalize_iccid_pairs(iccid_pairs)
out: dict[str, LegacyCardUsage] = {}
if not pairs:
return out
query_to_full: dict[str, tuple[str, int]] = {}
related_keys_by_full: dict[str, set[str]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
# tbl_card 主表同时查优先键和兜底键,结果选择时按 rank:
# 0=业务方完整 ICCID 精确命中,1=20 位卡在奇成主表只存 19 位时兜底。
query_to_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
query_to_full[i19] = (card_key, 1)
if not i20 or allow_i19_lookup:
related_keys_by_full.setdefault(card_key, set()).add(i19)
else:
related_keys_by_full.setdefault(card_key, set())
if i20:
related_keys_by_full[card_key].add(i20)
full_set = sorted(query_to_full.keys())
by_full = _build_card_lookup(pairs)
full_set = sorted(by_full.keys())
# tbl_card 主表用 iccid_mark IN 全长精确匹配;从表只在主表本身是 19 位时
# 按前缀兜底,避免同前缀 20 位卡互相串套餐。
sql = """
# ── 阶段1: tbl_card ───────────────────────────────────────────────────────
card_sql = """
SELECT
c.iccid_mark,
c.total_bytes_cnt,
COALESCE(sm.flow_add_discount, 0) AS flow_add_discount,
CASE WHEN cl.iccid_mark IS NOT NULL THEN 1 ELSE 0 END AS has_active_package
FROM tbl_card c
LEFT JOIN (
SELECT cl1.iccid_mark, cl1.meal_id
FROM tbl_card_life cl1
INNER JOIN (
SELECT iccid_mark, MAX(expire_date) AS max_expire
FROM tbl_card_life
WHERE status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
AND iccid_mark IN %s
GROUP BY iccid_mark
) latest
ON latest.iccid_mark = cl1.iccid_mark
AND latest.max_expire = cl1.expire_date
WHERE cl1.status = 1
AND COALESCE(CAST(cl1.type AS CHAR), '') <> '1'
) cl ON (
cl.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(cl.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN tbl_set_meal sm ON sm.id = cl.meal_id
WHERE c.iccid_mark IN %s
iccid_mark,
COALESCE(total_bytes_cnt, 0) AS total_bytes_cnt
FROM tbl_card
WHERE iccid_mark IN %s
"""
hits: dict[str, list[tuple[int, dict]]] = {}
card_rows = 0
card_batches = 0
with conn.cursor() as cur:
for batch in _chunks(full_set):
batch_keys = {query_to_full[x][0] for x in batch}
related_batch = sorted({x for key in batch_keys for x in related_keys_by_full[key]})
cur.execute(sql, (tuple(related_batch), tuple(batch)))
for batch in _chunks(full_set, batch_size):
card_batches += 1
cur.execute(card_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row["iccid_mark"]
if not legacy_iccid:
continue
matched = query_to_full.get(legacy_iccid)
matched = by_full.get(legacy_iccid)
if matched is None:
continue
card_key, rank = matched
hits.setdefault(card_key, []).append((rank, row))
card_rows += 1
timer.log("tbl_card", keys=len(full_set), batches=card_batches, rows=card_rows)
# 取最优命中行,构建 legacy_iccid 列表
card_data: dict[str, dict] = {} # card_key → best tbl_card row
for card_key, rows in hits.items():
best_rank = min(rank for rank, _row in rows)
row = next(row for rank, row in rows if rank == best_rank)
best_rank = min(r for r, _ in rows)
card_data[card_key] = next(row for r, row in rows if r == best_rank)
if not card_data:
return out
legacy_iccids = sorted({row["iccid_mark"] for row in card_data.values()})
# ── 阶段2: tbl_card_life 当前生效正式套餐(精确匹配,Python 选最新) ─────────
life_sql = """
SELECT
iccid_mark,
COALESCE(meal_id, '') AS meal_id
FROM tbl_card_life
WHERE iccid_mark IN %s
AND status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
ORDER BY iccid_mark ASC, expire_date DESC, id DESC
"""
current_meal_by_iccid: dict[str, str] = {} # legacy_iccid → meal_id
life_rows = 0
life_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
life_batches += 1
cur.execute(life_sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
life_rows += 1
if iccid not in current_meal_by_iccid:
current_meal_by_iccid[iccid] = row.get("meal_id") or ""
timer.log("tbl_card_life", keys=len(legacy_iccids), batches=life_batches, rows=life_rows)
# ── 阶段3: tbl_set_meal flow_add_discount(按 distinct meal_id 查) ─────────
meal_ids = sorted({mid for mid in current_meal_by_iccid.values() if mid})
discount_by_meal: dict[str, Decimal] = {}
if meal_ids:
meal_sql = "SELECT id, COALESCE(flow_add_discount, 0) AS flow_add_discount FROM tbl_set_meal WHERE id IN %s"
meal_batches = 0
with conn.cursor() as cur:
for batch in _chunks(meal_ids, batch_size):
meal_batches += 1
cur.execute(meal_sql, (tuple(batch),))
for row in cur.fetchall():
mid = str(row["id"] or "")
if mid:
discount_by_meal[mid] = _to_mb_decimal(row.get("flow_add_discount"))
timer.log("tbl_set_meal", meal_ids=len(meal_ids), batches=meal_batches)
else:
timer.log("tbl_set_meal", meal_ids=0, skipped=True)
# ── 阶段4: Python 组装 ────────────────────────────────────────────────────
for card_key, base_row in card_data.items():
legacy_iccid = base_row["iccid_mark"]
meal_id = current_meal_by_iccid.get(legacy_iccid, "")
has_active = bool(meal_id or legacy_iccid in current_meal_by_iccid)
discount = discount_by_meal.get(str(meal_id), Decimal("0")) if meal_id else Decimal("0")
out[card_key] = LegacyCardUsage(
iccid=card_key,
virtual_used_mb=_to_mb_decimal(row.get("total_bytes_cnt")),
flow_add_discount_pct=_to_mb_decimal(row.get("flow_add_discount")),
has_active_package=bool(row.get("has_active_package")),
virtual_used_mb=_to_mb_decimal(base_row.get("total_bytes_cnt")),
flow_add_discount_pct=discount,
has_active_package=has_active,
)
timer.log("assemble", cards=len(out))
return out
def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, LegacyCommissionAccount]:
def fetch_commission_accounts(
conn,
agent_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyCommissionAccount]:
"""查代理佣金账户。"""
timer = _StageTimer("fetch_commission_accounts")
agent_list = sorted({a for a in agent_ids if a})
out: dict[str, LegacyCommissionAccount] = {}
if not agent_list:
@@ -557,11 +651,15 @@ def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, Legac
FROM tbl_agent_commission_account
WHERE agent_id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(agent_list):
for batch in _chunks(agent_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
aid = row["agent_id"]
rows += 1
if not aid:
continue
out[aid] = LegacyCommissionAccount(
@@ -572,107 +670,69 @@ def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, Legac
total_commission_amount_fen=_to_fen(row.get("total_commision_amount")),
total_draw_amount_fen=_to_fen(row.get("total_draw_amount")),
)
timer.log("tbl_agent_commission_account", agents=len(agent_list), batches=batches, rows=rows)
return out
def fetch_card_meta(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> tuple[dict[str, LegacyCardMeta], set[str]]:
"""查每张卡的元数据(卡本体 + 当前生效套餐 + 虚拟号)
"""查每张卡的元数据,分阶段索引查询替代大 JOIN
阶段:
1. tbl_card — 卡基础字段 + 去重检测
2. tbl_vendor_category — 运营商分类名称(按 distinct category 查)
3. tbl_card_life — 当前生效正式套餐(Python 选 expire_date 最大,精确匹配)
4. tbl_set_meal + tbl_set_meal_series — 套餐系列(按 distinct meal_id 查)
5. tbl_virtual_number — 虚拟号(精确匹配,不做 LEFT() 前缀扫描)
6. Python 组装 LegacyCardMeta
匹配策略:
- tbl_card 主表同时查优先键和显式允许的兜底键:有 iccid_20 时优先
20 位精确命中;只有业务方也显式填了 iccid_19 时,才允许退回 19 位。
只填 iccid_20 的固定 20 位卡绝不拿 19 位前缀查主表。
- tbl_card_life / tbl_virtual_number 从表在主表本身是 19 位时才用
LEFT(iccid_mark, 19) 兜底,避免同前缀 20 位卡互相串数据
- tbl_card 主表同时查优先键和显式允许的兜底键;20 位精确命中优先
- 从表(card_life / virtual_number)用 tbl_card 实际返回的 iccid_mark 精确查询
- 不再使用 LEFT(vn.iccid_mark, 19) 前缀匹配,避免全表扫描
Args:
iccid_pairs: (iccid_19, iccid_20, allow_iccid_19_lookup) 列表。
Returns:
(metas, duplicate_iccids):
- metas: {完整 ICCID: LegacyCardMeta} 单匹结果
- duplicate_iccids: 一份业务方完整 ICCID 命中奇成 tbl_card 多条记录的 set,
这些卡不进 metas,sql_builder 会写 errors.csv 阻断
- metas: {完整 ICCID: LegacyCardMeta}
- duplicate_iccids: 一份业务方 ICCID 命中多条 tbl_card 记录的 set
"""
timer = _StageTimer("fetch_card_meta")
pairs = _normalize_iccid_pairs(iccid_pairs)
if not pairs:
return {}, set()
by_full: dict[str, tuple[str, int]] = {} # tbl_card 查询键 → (完整 ICCID, rank)
related_keys_by_full: dict[str, set[str]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
by_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
by_full[i19] = (card_key, 1)
if not i20 or allow_i19_lookup:
related_keys_by_full.setdefault(card_key, set()).add(i19)
else:
related_keys_by_full.setdefault(card_key, set())
if i20:
related_keys_by_full[card_key].add(i20)
by_full = _build_card_lookup(pairs)
full_set = sorted(by_full.keys())
sql = """
# ── 阶段1: tbl_card ───────────────────────────────────────────────────────
card_sql = """
SELECT
c.id AS card_id,
c.iccid_mark AS iccid,
COALESCE(c.account_id, '') AS account_id,
COALESCE(c.account_name, '') AS account_name,
COALESCE(c.category, 0) AS category,
COALESCE(vc.category_name, '') AS category_name,
COALESCE(c.agent_id, '') AS agent_id,
COALESCE(c.agent_name, '') AS agent_name,
COALESCE(c.phone, '') AS phone,
COALESCE(vn.vcode, '') AS vcode,
COALESCE(cl.meal_id, '') AS meal_id,
COALESCE(cl.meal_name, '') AS meal_name,
COALESCE(CAST(cl.meal_type AS CHAR), '') AS meal_type,
cl.expire_date AS expire_date,
COALESCE(sm.meal_series_id, '') AS series_id,
COALESCE(sms.name, '') AS series_name
FROM tbl_card c
LEFT JOIN tbl_vendor_category vc ON vc.category = c.category
LEFT JOIN tbl_virtual_number vn ON (
vn.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(vn.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN (
SELECT cl1.iccid_mark, cl1.meal_id, cl1.meal_name, cl1.type AS meal_type, cl1.expire_date
FROM tbl_card_life cl1
INNER JOIN (
SELECT iccid_mark, MAX(expire_date) AS max_expire
FROM tbl_card_life
WHERE status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
AND iccid_mark IN %s
GROUP BY iccid_mark
) latest
ON latest.iccid_mark = cl1.iccid_mark
AND latest.max_expire = cl1.expire_date
WHERE cl1.status = 1
AND COALESCE(CAST(cl1.type AS CHAR), '') <> '1'
) cl ON (
cl.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(cl.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN tbl_set_meal sm ON sm.id = cl.meal_id
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE c.iccid_mark IN %s
id AS card_id,
iccid_mark,
COALESCE(account_id, '') AS account_id,
COALESCE(account_name, '') AS account_name,
COALESCE(category, 0) AS category,
COALESCE(agent_id, '') AS agent_id,
COALESCE(agent_name, '') AS agent_name,
COALESCE(phone, '') AS phone
FROM tbl_card
WHERE iccid_mark IN %s
"""
# 收集每个完整 ICCID 命中的奇成行(可能多条 = 脏数据)
hits: dict[str, list[tuple[int, dict]]] = {}
card_rows = 0
card_batches = 0
with conn.cursor() as cur:
for batch in _chunks(full_set):
batch_keys = {by_full[x][0] for x in batch}
related_batch = sorted({x for key in batch_keys for x in related_keys_by_full[key]})
cur.execute(sql, (tuple(related_batch), tuple(batch)))
for batch in _chunks(full_set, batch_size):
card_batches += 1
cur.execute(card_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row["iccid"]
legacy_iccid = row["iccid_mark"]
if not legacy_iccid:
continue
matched = by_full.get(legacy_iccid)
@@ -680,45 +740,162 @@ def fetch_card_meta(
continue
card_key, rank = matched
hits.setdefault(card_key, []).append((rank, row))
card_rows += 1
timer.log("tbl_card", keys=len(full_set), batches=card_batches, rows=card_rows)
metas: dict[str, LegacyCardMeta] = {}
# 去重:同一完整 ICCID 命中多条不同 card_id → duplicates
metas_base: dict[str, dict] = {} # card_key → best tbl_card row
duplicates: set[str] = set()
categories: set[int] = set()
for card_key, rows in hits.items():
best_rank = min(rank for rank, _row in rows)
best_rows = [row for rank, row in rows if rank == best_rank]
# 同一张 tbl_card 可能因 card_life / virtual_number 前缀兜底 JOIN 出多行,
# 这不是主表重复;只有不同 card_id 才算同一业务 ICCID 命中多张卡。
best_rank = min(r for r, _ in rows)
best_rows = [row for r, row in rows if r == best_rank]
rows_by_card_id: dict[str, dict] = {}
for row in best_rows:
rows_by_card_id[str(row.get("card_id") or row.get("iccid") or "")] = row
rows_by_card_id[str(row.get("card_id") or row.get("iccid_mark") or "")] = row
if len(rows_by_card_id) > 1:
duplicates.add(card_key)
continue
row = next(iter(rows_by_card_id.values()))
expire = row.get("expire_date")
metas_base[card_key] = row
categories.add(int(row.get("category") or 0))
if not metas_base:
return {}, duplicates
# legacy_iccids: tbl_card 实际存储的 iccid_mark 值(用于从表精确查询)
legacy_iccids = sorted({row["iccid_mark"] for row in metas_base.values()})
# ── 阶段2: tbl_vendor_category(distinct category,通常极少几十个) ─────────
category_names: dict[int, str] = {}
cat_batches = 0
if categories:
cat_sql = """
SELECT category, COALESCE(category_name, '') AS category_name
FROM tbl_vendor_category
WHERE category IN %s
"""
cat_list = sorted(categories)
with conn.cursor() as cur:
for batch in _chunks(cat_list, batch_size):
cat_batches += 1
cur.execute(cat_sql, (tuple(batch),))
for row in cur.fetchall():
category_names[int(row["category"])] = row.get("category_name") or ""
timer.log("tbl_vendor_category", categories=len(categories), batches=cat_batches)
# ── 阶段3: tbl_card_life 当前生效正式套餐(精确匹配,Python 选最新) ─────────
# 精确用 legacy_iccid 查询;不再做 LEFT(iccid_mark, 19) 前缀扫描
life_sql = """
SELECT
iccid_mark,
COALESCE(meal_id, '') AS meal_id,
COALESCE(meal_name, '') AS meal_name,
COALESCE(CAST(type AS CHAR), '') AS meal_type,
expire_date
FROM tbl_card_life
WHERE iccid_mark IN %s
AND status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
ORDER BY iccid_mark ASC, expire_date DESC, id DESC
"""
current_meals: dict[str, dict] = {} # legacy_iccid → 当前生效套餐行
meal_ids: set[str] = set()
life_rows = 0
life_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
life_batches += 1
cur.execute(life_sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
life_rows += 1
if iccid not in current_meals:
current_meals[iccid] = row
if row.get("meal_id"):
meal_ids.add(row["meal_id"])
timer.log("tbl_card_life", keys=len(legacy_iccids), batches=life_batches, rows=life_rows, found=len(current_meals))
# ── 阶段4: tbl_set_meal + tbl_set_meal_series(按 distinct meal_id) ───────
meal_series: dict[str, tuple[str, str]] = {} # meal_id → (series_id, series_name)
meal_batches = 0
if meal_ids:
meal_sql = """
SELECT
sm.id AS meal_id,
COALESCE(sm.meal_series_id, '') AS series_id,
COALESCE(sms.name, '') AS series_name
FROM tbl_set_meal sm
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE sm.id IN %s
"""
meal_id_list = sorted(meal_ids)
with conn.cursor() as cur:
for batch in _chunks(meal_id_list, batch_size):
meal_batches += 1
cur.execute(meal_sql, (tuple(batch),))
for row in cur.fetchall():
mid = str(row["meal_id"] or "")
if mid:
meal_series[mid] = (row.get("series_id") or "", row.get("series_name") or "")
timer.log("tbl_set_meal", meal_ids=len(meal_ids), batches=meal_batches)
# ── 阶段5: tbl_virtual_number(精确匹配,不做前缀扫描) ─────────────────────
virtual_nos: dict[str, str] = {} # legacy_iccid → vcode
vn_rows = 0
vn_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
vn_batches += 1
cur.execute(
"SELECT iccid_mark, COALESCE(vcode, '') AS vcode FROM tbl_virtual_number WHERE iccid_mark IN %s",
(tuple(batch),),
)
for row in cur.fetchall():
iccid = row["iccid_mark"]
vn_rows += 1
if iccid:
virtual_nos[iccid] = row.get("vcode") or ""
timer.log("tbl_virtual_number", keys=len(legacy_iccids), batches=vn_batches, rows=vn_rows)
# ── 阶段6: Python 组装 ────────────────────────────────────────────────────
metas: dict[str, LegacyCardMeta] = {}
for card_key, base_row in metas_base.items():
legacy_iccid = base_row["iccid_mark"]
meal_row = current_meals.get(legacy_iccid, {})
expire = meal_row.get("expire_date")
expire_str = expire.strftime("%Y-%m-%d %H:%M:%S") if expire else None
meal_id = meal_row.get("meal_id") or ""
series_id, series_name = meal_series.get(str(meal_id), ("", ""))
metas[card_key] = LegacyCardMeta(
legacy_raw_iccid=row["iccid"],
account_id=row.get("account_id") or "",
account_name=row.get("account_name") or "",
category=int(row.get("category") or 0),
category_name=row.get("category_name") or "",
agent_id=row.get("agent_id") or "",
agent_name=row.get("agent_name") or "",
msisdn=row.get("phone") or "",
virtual_no=row.get("vcode") or "",
current_meal_id=row.get("meal_id") or "",
current_meal_name=row.get("meal_name") or "",
current_meal_type=str(row.get("meal_type") or ""),
legacy_raw_iccid=legacy_iccid,
account_id=base_row.get("account_id") or "",
account_name=base_row.get("account_name") or "",
category=int(base_row.get("category") or 0),
category_name=category_names.get(int(base_row.get("category") or 0), ""),
agent_id=base_row.get("agent_id") or "",
agent_name=base_row.get("agent_name") or "",
msisdn=base_row.get("phone") or "",
virtual_no=virtual_nos.get(legacy_iccid, ""),
current_meal_id=meal_id,
current_meal_name=meal_row.get("meal_name") or "",
current_meal_type=str(meal_row.get("meal_type") or ""),
current_expire_date=expire_str,
current_series_id=row.get("series_id") or "",
current_series_name=row.get("series_name") or "",
current_series_id=series_id,
current_series_name=series_name,
)
timer.log("assemble", cards=len(metas), duplicates=len(duplicates))
return metas, duplicates
def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
def fetch_meal_meta(
conn,
meal_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyMealMeta]:
"""查套餐 + 系列元数据(供 scan_legacy 收集映射 legacy_* 字段)。"""
timer = _StageTimer("fetch_meal_meta")
id_list = sorted({m for m in meal_ids if m})
out: dict[str, LegacyMealMeta] = {}
if not id_list:
@@ -733,11 +910,15 @@ def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE sm.id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(id_list):
for batch in _chunks(id_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
mid = row["meal_id"]
rows += 1
if not mid:
continue
out[mid] = LegacyMealMeta(
@@ -746,11 +927,17 @@ def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
series_id=row.get("series_id") or "",
series_name=row.get("series_name") or "",
)
timer.log("tbl_set_meal", meal_ids=len(id_list), batches=batches, rows=rows)
return out
def fetch_agent_balances(conn, agent_ids: Iterable[str]) -> dict[str, LegacyAgentBalance]:
def fetch_agent_balances(
conn,
agent_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyAgentBalance]:
"""查代理主钱包余额(tbl_agent.balance_money,元单位)。"""
timer = _StageTimer("fetch_agent_balances")
agent_list = sorted({a for a in agent_ids if a})
out: dict[str, LegacyAgentBalance] = {}
if not agent_list:
@@ -761,12 +948,17 @@ def fetch_agent_balances(conn, agent_ids: Iterable[str]) -> dict[str, LegacyAgen
FROM tbl_agent
WHERE id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(agent_list):
for batch in _chunks(agent_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
aid = row["id"]
rows += 1
if not aid:
continue
out[aid] = LegacyAgentBalance(agent_id=aid, balance_fen=_to_fen(row.get("balance_money")))
timer.log("tbl_agent", agents=len(agent_list), batches=batches, rows=rows)
return out

View File

@@ -139,6 +139,7 @@ class AgentMapping:
legacy_agent_id: str
legacy_agent_name: str
target_shop_code: Optional[str] # legacy_agent 模式下允许 null,表示该代理资产进平台库存
no_downtime: bool = False # 迁移时免停机:有生效套餐的卡直接以 network_status=1 导入
@dataclass
@@ -166,6 +167,11 @@ class Mapping:
return None
return self.series.get(str(legacy_series_id).strip())
def is_no_downtime_agent(self, legacy_agent_id: str) -> bool:
"""判断代理是否配置了免停机迁移。"""
agent = self.agents.get(str(legacy_agent_id or "").strip())
return agent.no_downtime if agent else False
def lookup_agent_shop_code(self, legacy_agent_id: str) -> Optional[str]:
"""按 agent_id 查 target_shop_code,空字符串视为 None(进平台库存)。"""
if not legacy_agent_id:
@@ -260,6 +266,7 @@ def load_mapping(config_dir: Path) -> Mapping:
legacy_agent_id=str(item["legacy_agent_id"]).strip(),
legacy_agent_name=str(item.get("legacy_agent_name") or ""),
target_shop_code=_opt_shop_code(item.get("target_shop_code"), "agents[].target_shop_code"),
no_downtime=bool(item.get("no_downtime", False)),
)
m.agents[a.legacy_agent_id] = a
@@ -536,6 +543,7 @@ def _append_agents(lines: list[str], mapping: Mapping) -> None:
f" - legacy_agent_id: {_yaml_scalar(item.legacy_agent_id)}",
f" legacy_agent_name: {_yaml_scalar(item.legacy_agent_name)}",
f" target_shop_code: {_yaml_scalar(item.target_shop_code)}",
f" no_downtime: {_yaml_scalar(item.no_downtime)}",
])

View File

@@ -621,6 +621,20 @@ def _write_iot_cards(
card_status = 2 if shop_code else 1
shop_id_expr = _shop_id_sub(shop_code)
# 免停机迁移:代理配置 no_downtime=true 且卡在奇成有当前生效正式套餐
# finalize 保证最后执行,此时 step2 套餐已落库,轮询系统不会在此之前触碰这张卡
is_no_downtime = bool(meta.current_meal_id) and mapping.is_no_downtime_agent(meta.agent_id)
if is_no_downtime:
activation_status = 1
real_name_status = 1 # 卡在线说明已实名,写 1 防止 EvaluateAndAct 因 not_realname 停机
network_status = 1
stop_reason_expr = "NULL"
else:
activation_status = 0
real_name_status = 0
network_status = 0
stop_reason_expr = _sql_str(_initial_polling_stop_reason(c))
f.write(
"INSERT INTO tb_iot_card (\n"
" iccid, iccid_19, iccid_20,\n"
@@ -636,9 +650,9 @@ def _write_iot_cards(
f" {_sql_str(c.iccid_full)}, {_sql_str(c.iccid_19)}, {iccid_20_expr},\n"
f" {carrier.target_carrier_id}, {_sql_str(carrier.target_carrier_type)}, {_sql_str(carrier.target_carrier_name)},\n"
f" {_sql_str(card_category)}, {_sql_str(mapping.migration_batch_no)},\n"
f" {card_status}, 0, 0, 0,\n"
f" {card_status}, {activation_status}, {real_name_status}, {network_status},\n"
f" 1, 1, {is_standalone},\n"
f" 'after_order', {_sql_str(_initial_polling_stop_reason(c))},\n"
f" 'after_order', {stop_reason_expr},\n"
f" {_sql_str(meta.msisdn) if meta.msisdn else 'NULL'}, "
f"{_sql_str(card_virtual_no) if card_virtual_no else 'NULL'}, "
f"{shop_id_expr}, {series_id_expr},\n"
@@ -1077,7 +1091,6 @@ def write_step2(
cards: list[CardRow],
devices: list[DeviceRow],
mapping: Mapping,
card_metas: dict[str, LegacyCardMeta],
usage_snapshots: dict[str, LegacyCardUsage],
package_lifecycles: dict[str, list[LegacyPackageLifecycle]],
commission_snapshots: dict[str, LegacyCommissionAccount],

View File

@@ -4,7 +4,7 @@
工作流:
1. 读 mapping.yaml(必须先跑过 scan_legacy.py)
2. 读 cards.csv / devices.csv
3. 连奇成查每张卡的元数据(category/agent_id/msisdn/virtual_no)
3. 连奇成查每张卡的元数据(category/agent_id/msisdn/virtual_no/当前生效套餐)
4. 应用 mapping 把 legacy_* 翻译成 target_*,自动决定归属
5. 生成 output/step1_*.sql + errors.csv + summary.txt
@@ -14,11 +14,13 @@
用法:
python3 migrate_assets.py [--config-dir config] [--resources-dir resources] [--output-dir output]
[--batch-size 1000]
"""
from __future__ import annotations
import argparse
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
@@ -31,6 +33,7 @@ def main() -> int:
parser.add_argument("--config-dir", default="config", help="yaml 配置目录")
parser.add_argument("--resources-dir", default="resources", help="CSV 资源目录")
parser.add_argument("--output-dir", default="output", help="SQL/CSV 输出目录")
parser.add_argument("--batch-size", type=int, default=legacy_query.BATCH_SIZE, help="老库批量查询大小")
args = parser.parse_args()
base = Path(__file__).resolve().parent
@@ -38,16 +41,26 @@ def main() -> int:
resources_dir = (base / args.resources_dir).resolve()
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
batch_size = args.batch_size
t0 = time.perf_counter()
mapping = mapping_loader.load_mapping(config_dir)
cards, devices, errors = csv_loader.load_assets(resources_dir, mapping)
print(f"[migrate] 加载 CSV: cards={len(cards)} devices={len(devices)} elapsed={time.perf_counter()-t0:.1f}s")
# 连奇成查卡元数据(运营商分类、agent_id、msisdn、virtual_no、当前生效套餐)
iccid_pairs = [(c.iccid_19, c.iccid_20, c.allow_iccid_19_lookup) for c in cards]
t1 = time.perf_counter()
dsn = mapping_loader.load_legacy_dsn(config_dir)
with legacy_query.connect_readonly(dsn) as conn:
card_metas, dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs)
print(f"[migrate] 连接老库: elapsed={time.perf_counter()-t1:.1f}s")
t2 = time.perf_counter()
card_metas, dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_card_meta: cards={len(card_metas)} duplicates={len(dup_iccids)} elapsed={time.perf_counter()-t2:.1f}s")
t3 = time.perf_counter()
sql_builder.write_step1(
output_dir,
cards=cards,
@@ -57,8 +70,10 @@ def main() -> int:
duplicate_iccids=dup_iccids,
errors=errors,
)
print(f"[migrate] SQL 生成: elapsed={time.perf_counter()-t3:.1f}s")
print(f"完成:卡 {len(cards)} 张,设备 {len(devices)} 台,异常 {len(errors)}")
total = time.perf_counter() - t0
print(f"完成:卡 {len(cards)} 张,设备 {len(devices)} 台,异常 {len(errors)} 行,总耗时 {total:.1f}s")
print(f"SQL 输出: {output_dir}")
return 0

View File

@@ -5,17 +5,19 @@
工作流:
1. 读 mapping.yaml + cards.csv
2. 连奇成查:卡元数据(card_meta) + 累计流量(card_usage) + 代理佣金/主钱包余额
2. 连奇成查:累计流量(card_usage) + 套餐生命周期 + 代理佣金/主钱包余额
3. 卡 → 当前生效套餐 → mapping.packages → target_package_id
4. 生成 step2_*.sql
用法:
python3 migrate_runtime.py [--config-dir config] [--resources-dir resources] [--output-dir output]
[--batch-size 1000]
"""
from __future__ import annotations
import argparse
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
@@ -28,6 +30,7 @@ def main() -> int:
parser.add_argument("--config-dir", default="config", help="yaml 配置目录")
parser.add_argument("--resources-dir", default="resources", help="CSV 资源目录")
parser.add_argument("--output-dir", default="output", help="SQL/CSV 输出目录")
parser.add_argument("--batch-size", type=int, default=legacy_query.BATCH_SIZE, help="老库批量查询大小")
args = parser.parse_args()
base = Path(__file__).resolve().parent
@@ -35,9 +38,13 @@ def main() -> int:
resources_dir = (base / args.resources_dir).resolve()
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
batch_size = args.batch_size
t0 = time.perf_counter()
mapping = mapping_loader.load_mapping(config_dir)
cards, devices, errors = csv_loader.load_assets(resources_dir, mapping)
print(f"[migrate] 加载 CSV: cards={len(cards)} devices={len(devices)} elapsed={time.perf_counter()-t0:.1f}s")
if errors:
sql_builder.write_runtime_input_errors(output_dir, errors)
print(f"cards.csv/devices.csv 存在 {len(errors)} 个基础错误,已写入 {output_dir / 'errors.csv'}", file=sys.stderr)
@@ -46,27 +53,39 @@ def main() -> int:
iccid_pairs = [(c.iccid_19, c.iccid_20, c.allow_iccid_19_lookup) for c in cards]
agent_ids = [a.legacy_agent_id for a in mapping.agents.values() if (a.target_shop_code or "").strip()]
t1 = time.perf_counter()
dsn = mapping_loader.load_legacy_dsn(config_dir)
with legacy_query.connect_readonly(dsn) as conn:
card_metas, _dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs)
usage_snapshots = legacy_query.fetch_card_usage(conn, iccid_pairs)
package_lifecycles = legacy_query.fetch_package_lifecycles(conn, iccid_pairs)
commission_snapshots = legacy_query.fetch_commission_accounts(conn, agent_ids) if agent_ids else {}
agent_balances = legacy_query.fetch_agent_balances(conn, agent_ids) if agent_ids else {}
print(f"[migrate] 连接老库: elapsed={time.perf_counter()-t1:.1f}s")
t2 = time.perf_counter()
usage_snapshots = legacy_query.fetch_card_usage(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_card_usage: cards={len(usage_snapshots)} elapsed={time.perf_counter()-t2:.1f}s")
t3 = time.perf_counter()
package_lifecycles = legacy_query.fetch_package_lifecycles(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_package_lifecycles: cards={len(package_lifecycles)} elapsed={time.perf_counter()-t3:.1f}s")
t4 = time.perf_counter()
commission_snapshots = legacy_query.fetch_commission_accounts(conn, agent_ids, batch_size=batch_size) if agent_ids else {}
agent_balances = legacy_query.fetch_agent_balances(conn, agent_ids, batch_size=batch_size) if agent_ids else {}
print(f"[migrate] fetch_agent_data: agents={len(agent_ids)} elapsed={time.perf_counter()-t4:.1f}s")
t5 = time.perf_counter()
warnings = sql_builder.write_step2(
output_dir,
cards=cards,
devices=devices,
mapping=mapping,
card_metas=card_metas,
usage_snapshots=usage_snapshots,
package_lifecycles=package_lifecycles,
commission_snapshots=commission_snapshots,
agent_balances=agent_balances,
)
print(f"[migrate] SQL 生成: elapsed={time.perf_counter()-t5:.1f}s")
print(f"完成:卡 {len(cards)} 张,代理 {len(agent_ids)} 个,警告 {len(warnings)}")
total = time.perf_counter() - t0
print(f"完成:卡 {len(cards)} 张,代理 {len(agent_ids)} 个,警告 {len(warnings)} 条,总耗时 {total:.1f}s")
print(f"SQL 输出: {output_dir}")
return 0

View File

@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""卡删除脚本:读取 ICCID 列表,生成数据库删除 SQL 和 Redis 轮询清理命令。
用法:
python3 remove_cards.py --input <csv文件> [--output-dir output]
CSV 格式: 纯文本,每行一个 ICCID首行可有 "iccid" 表头(自动跳过)
输出:
output/remove_cards_db.sql -- 数据库软删 + 标识符硬删 SQL事务封装
output/remove_cards_redis_gen.sql -- 在数据库执行后把输出通过 redis-cli 清理轮询队列
"""
from __future__ import annotations
import argparse
import sys
from datetime import datetime
from pathlib import Path
def load_iccids(csv_path: Path) -> list[str]:
"""读取 CSV 文件中的 ICCID 列表,自动跳过空行和表头。"""
iccids: list[str] = []
with csv_path.open(encoding="utf-8") as f:
for lineno, raw in enumerate(f, 1):
line = raw.strip()
if not line:
continue
# 跳过纯字母表头(如 "iccid"
if lineno == 1 and not line.isdigit() and not line.startswith("8"):
print(f"[remove] 跳过表头: {line!r}", file=sys.stderr)
continue
iccids.append(line)
return iccids
def sql_in_list(iccids: list[str]) -> str:
"""生成 SQL IN 列表字符串,如 '898...', '898...'"""
return ", ".join(f"'{i}'" for i in iccids)
def generate_db_sql(iccids: list[str], generated_at: str) -> str:
in_list = sql_in_list(iccids)
count = len(iccids)
return f"""\
-- ============================================================
-- 迁移卡删除脚本 (数据库部分)
-- 生成时间: {generated_at}
-- 卡数量 : {count}
-- 执行前请先用 remove_cards_redis_gen.sql 生成 Redis 清理命令并执行
-- 建议执行: psql ... -1 -f <此文件>
-- ============================================================
BEGIN;
-- 临时表:本次要删除的卡
CREATE TEMP TABLE tmp_remove_cards AS
SELECT id, iccid
FROM tb_iot_card
WHERE iccid IN ({in_list});
DO $$ BEGIN
RAISE NOTICE '本次将删除 % 张卡(共输入 {count} 个 ICCID', (SELECT COUNT(*) FROM tmp_remove_cards);
END $$;
-- 1. 删除套餐使用记录
DELETE FROM tb_package_usage
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 2. 删除订单
DELETE FROM tb_order
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 3. 删除分配记录
DELETE FROM tb_asset_allocation_record
WHERE asset_type = 'iot_card'
AND asset_id IN (SELECT id FROM tmp_remove_cards);
-- 4. 删除资产钱包
DELETE FROM tb_asset_wallet
WHERE resource_type = 'iot_card'
AND resource_id IN (SELECT id FROM tmp_remove_cards);
-- 5. 删除设备绑卡关系
DELETE FROM tb_device_sim_binding
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 6. 删除资产标识符
DELETE FROM tb_asset_identifier
WHERE asset_type = 'iot_card'
AND asset_id IN (SELECT id FROM tmp_remove_cards);
-- 7. 删除卡本体(最后执行)
DELETE FROM tb_iot_card
WHERE id IN (SELECT id FROM tmp_remove_cards);
COMMIT;
"""
def generate_redis_gen_sql(iccids: list[str], generated_at: str) -> str:
in_list = sql_in_list(iccids)
count = len(iccids)
return f"""\
-- ============================================================
-- 轮询队列清理命令生成 SQL
-- 生成时间: {generated_at}
-- 卡数量 : {count}
-- 执行方式:
-- psql ... -t -A -c "$(cat remove_cards_redis_gen.sql)" | redis-cli --pipe
-- 或:
-- psql ... -t -A -f remove_cards_redis_gen.sql | redis-cli
-- ============================================================
WITH cards AS (
SELECT id FROM tb_iot_card
WHERE iccid IN ({in_list})
),
task_types(task_type) AS (
VALUES
('polling:realname'),
('polling:carddata'),
('polling:package'),
('polling:protect'),
('polling:card_status')
)
SELECT format('ZREM polling:shard:%s:queue:%s %s', id % 16, task_type, id)
FROM cards CROSS JOIN task_types
UNION ALL
SELECT format('DEL polling:card:%s', id)
FROM cards
UNION ALL
SELECT format('LREM polling:manual:%s 0 %s', task_type, id)
FROM cards CROSS JOIN task_types
UNION ALL
SELECT format('SREM polling:manual:dedupe:%s %s', task_type, id)
FROM cards CROSS JOIN task_types;
"""
def main() -> int:
parser = argparse.ArgumentParser(description="迁移卡删除脚本:生成删除 SQL 和 Redis 清理命令")
parser.add_argument("--input", required=True, help="ICCID 列表 CSV 文件路径")
parser.add_argument("--output-dir", default="output", help="输出目录(默认 output")
args = parser.parse_args()
base = Path(__file__).resolve().parent
input_path = Path(args.input)
if not input_path.is_absolute():
input_path = base / input_path
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
if not input_path.exists():
print(f"[remove] 错误: 找不到输入文件 {input_path}", file=sys.stderr)
return 1
iccids = load_iccids(input_path)
if not iccids:
print("[remove] 错误: CSV 中没有找到有效的 ICCID", file=sys.stderr)
return 1
print(f"[remove] 读取到 {len(iccids)} 个 ICCID开始生成 SQL...")
generated_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
db_sql_path = output_dir / "remove_cards_db.sql"
redis_sql_path = output_dir / "remove_cards_redis_gen.sql"
db_sql_path.write_text(generate_db_sql(iccids, generated_at), encoding="utf-8")
redis_sql_path.write_text(generate_redis_gen_sql(iccids, generated_at), encoding="utf-8")
print(f"[remove] 生成完成:")
print(f" 数据库删除 SQL : {db_sql_path}")
print(f" Redis 清理 SQL : {redis_sql_path}")
print()
print("执行步骤:")
print(" 1. 先执行 Redis 清理(在删除卡数据之前,以便 SQL 还能查到卡 ID:")
print(f" psql <连接参数> -t -A -f {redis_sql_path} | redis-cli")
print(" 2. 再执行数据库删除:")
print(f" psql <连接参数> -1 -f {db_sql_path}")
return 0
if __name__ == "__main__":
sys.exit(main())