10 Commits

Author SHA1 Message Date
d2e08dbbec skill提交
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 6m22s
2026-07-11 15:32:56 +09:00
026d4908d8 删除
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 53s
2026-07-11 12:28:38 +09:00
31232ea899 优化迁移脚本速度
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 53s
2026-07-10 13:10:00 +09:00
b38df737e1 先短暂去除限制,上传迁移脚本
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m51s
2026-07-09 18:23:29 +09:00
346156ee9b 卡只允许支付宝支付,设备只允许微信支付,钱包充值也遵循这个规则
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m53s
2026-07-03 10:26:48 +09:00
0d79130e07 入参
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m46s
2026-07-02 17:02:13 +09:00
b3fb8c7a82 资产详情新增两个字段
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
2026-07-02 16:49:50 +09:00
44fb21eb6a 修复导入重试的问题
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m53s
2026-07-02 15:13:22 +09:00
8f738ffbe8 导入的问题修复
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 7m51s
2026-07-02 13:04:36 +09:00
6db152bb2f chore: 安装 ponytail lazy senior dev 规则 2026-07-01 12:26:19 +09:00
50 changed files with 1155 additions and 714 deletions

30
.agents/rules/ponytail.md Normal file
View File

@@ -0,0 +1,30 @@
# Ponytail, lazy senior dev mode
You are a lazy senior developer. Lazy means efficient, not careless. The best code is the code never written.
Before writing any code, stop at the first rung that holds:
1. Does this need to be built at all? (YAGNI)
2. Does it already exist in this codebase? Reuse the helper, util, or pattern that's already here, don't re-write it.
3. Does the standard library already do this? Use it.
4. Does a native platform feature cover it? Use it.
5. Does an already-installed dependency solve it? Use it.
6. Can this be one line? Make it one line.
7. Only then: write the minimum code that works.
The ladder runs after you understand the problem, not instead of it: read the task and the code it touches, trace the real flow end to end, then climb.
Bug fix = root cause, not symptom: a report names a symptom. Grep every caller of the function you touch and fix the shared function once — one guard there is a smaller diff than one per caller, and patching only the path the ticket names leaves a sibling caller still broken.
Rules:
- No abstractions that weren't explicitly requested.
- No new dependency if it can be avoided.
- No boilerplate nobody asked for.
- Deletion over addition. Boring over clever. Fewest files possible.
- Shortest working diff wins, but only once you understand the problem. The smallest change in the wrong place isn't lazy, it's a second bug.
- Question complex requests: "Do you actually need X, or does Y cover it?"
- Pick the edge-case-correct option when two stdlib approaches are the same size, lazy means less code, not the flimsier algorithm.
- Mark intentional simplifications with a `ponytail:` comment. If the shortcut has a known ceiling (global lock, O(n²) scan, naive heuristic), the comment names the ceiling and the upgrade path.
Not lazy about: understanding the problem (read it fully and trace the real flow before picking a rung, a small diff you don't understand is just laziness dressed up as efficiency), input validation at trust boundaries, error handling that prevents data loss, security, accessibility, the calibration real hardware needs (the platform is never the spec ideal, a clock drifts, a sensor reads off), anything explicitly requested. Lazy code without its check is unfinished: non-trivial logic leaves ONE runnable check behind, the smallest thing that fails if the logic breaks (an assert-based demo/self-check or one small test file; no frameworks, no fixtures). Trivial one-liners need no test.

View File

@@ -1,10 +1,7 @@
---
name: grill-me
description: Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
description: A relentless interview to sharpen a plan or design.
disable-model-invocation: true
---
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time.
If a question can be answered by exploring the codebase, explore the codebase instead.
Run a `/grilling` session.

View File

@@ -1,47 +0,0 @@
# ADR Format
ADRs live in `docs/adr/` and use sequential numbering: `0001-slug.md`, `0002-slug.md`, etc.
Create the `docs/adr/` directory lazily — only when the first ADR is needed.
## Template
```md
# {Short title of the decision}
{1-3 sentences: what's the context, what did we decide, and why.}
```
That's it. An ADR can be a single paragraph. The value is in recording *that* a decision was made and *why* — not in filling out sections.
## Optional sections
Only include these when they add genuine value. Most ADRs won't need them.
- **Status** frontmatter (`proposed | accepted | deprecated | superseded by ADR-NNNN`) — useful when decisions are revisited
- **Considered Options** — only when the rejected alternatives are worth remembering
- **Consequences** — only when non-obvious downstream effects need to be called out
## Numbering
Scan `docs/adr/` for the highest existing number and increment by one.
## When to offer an ADR
All three of these must be true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will look at the code and wonder "why on earth did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If a decision is easy to reverse, skip it — you'll just reverse it. If it's not surprising, nobody will wonder why. If there was no real alternative, there's nothing to record beyond "we did the obvious thing."
### What qualifies
- **Architectural shape.** "We're using a monorepo." "The write model is event-sourced, the read model is projected into Postgres."
- **Integration patterns between contexts.** "Ordering and Billing communicate via domain events, not synchronous HTTP."
- **Technology choices that carry lock-in.** Database, message bus, auth provider, deployment target. Not every library — just the ones that would take a quarter to swap out.
- **Boundary and scope decisions.** "Customer data is owned by the Customer context; other contexts reference it by ID only." The explicit no-s are as valuable as the yes-s.
- **Deliberate deviations from the obvious path.** "We're using manual SQL instead of an ORM because X." Anything where a reasonable reader would assume the opposite. These stop the next engineer from "fixing" something that was deliberate.
- **Constraints not visible in the code.** "We can't use AWS because of compliance requirements." "Response times must be under 200ms because of the partner API contract."
- **Rejected alternatives when the rejection is non-obvious.** If you considered GraphQL and picked REST for subtle reasons, record it — otherwise someone will suggest GraphQL again in six months.

View File

@@ -1,60 +0,0 @@
# CONTEXT.md Format
## Structure
```md
# {Context Name}
{One or two sentence description of what this context is and why it exists.}
## Language
**Order**:
{A one or two sentence description of the term}
_Avoid_: Purchase, transaction
**Invoice**:
A request for payment sent to a customer after delivery.
_Avoid_: Bill, payment request
**Customer**:
A person or organization that places orders.
_Avoid_: Client, buyer, account
```
## Rules
- **Be opinionated.** When multiple words exist for the same concept, pick the best one and list the others under `_Avoid_`.
- **Keep definitions tight.** One or two sentences max. Define what it IS, not what it does.
- **Only include terms specific to this project's context.** General programming concepts (timeouts, error types, utility patterns) don't belong even if the project uses them extensively. Before adding a term, ask: is this a concept unique to this context, or a general programming concept? Only the former belongs.
- **Group terms under subheadings** when natural clusters emerge. If all terms belong to a single cohesive area, a flat list is fine.
## Single vs multi-context repos
**Single context (most repos):** One `CONTEXT.md` at the repo root.
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts, where they live, and how they relate to each other:
```md
# Context Map
## Contexts
- [Ordering](./src/ordering/CONTEXT.md) — receives and tracks customer orders
- [Billing](./src/billing/CONTEXT.md) — generates invoices and processes payments
- [Fulfillment](./src/fulfillment/CONTEXT.md) — manages warehouse picking and shipping
## Relationships
- **Ordering → Fulfillment**: Ordering emits `OrderPlaced` events; Fulfillment consumes them to start picking
- **Fulfillment → Billing**: Fulfillment emits `ShipmentDispatched` events; Billing consumes them to generate invoices
- **Ordering ↔ Billing**: Shared types for `CustomerId` and `Money`
```
The skill infers which structure applies:
- If `CONTEXT-MAP.md` exists, read it to find contexts
- If only a root `CONTEXT.md` exists, single context
- If neither exists, create a root `CONTEXT.md` lazily when the first term is resolved
When multiple contexts exist, infer which one the current topic relates to. If unclear, ask.

View File

@@ -1,88 +1,7 @@
---
name: grill-with-docs
description: Grilling session that challenges your plan against the existing domain model, sharpens terminology, and updates documentation (CONTEXT.md, ADRs) inline as decisions crystallise. Use when user wants to stress-test a plan against their project's language and documented decisions.
description: A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
disable-model-invocation: true
---
<what-to-do>
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time, waiting for feedback on each question before continuing.
If a question can be answered by exploring the codebase, explore the codebase instead.
</what-to-do>
<supporting-info>
## Domain awareness
During codebase exploration, also look for existing documentation:
### File structure
Most repos have a single context:
```
/
├── CONTEXT.md
├── docs/
│ └── adr/
│ ├── 0001-event-sourced-orders.md
│ └── 0002-postgres-for-write-model.md
└── src/
```
If a `CONTEXT-MAP.md` exists at the root, the repo has multiple contexts. The map points to where each one lives:
```
/
├── CONTEXT-MAP.md
├── docs/
│ └── adr/ ← system-wide decisions
├── src/
│ ├── ordering/
│ │ ├── CONTEXT.md
│ │ └── docs/adr/ ← context-specific decisions
│ └── billing/
│ ├── CONTEXT.md
│ └── docs/adr/
```
Create files lazily — only when you have something to write. If no `CONTEXT.md` exists, create one when the first term is resolved. If no `docs/adr/` exists, create it when the first ADR is needed.
## During the session
### Challenge against the glossary
When the user uses a term that conflicts with the existing language in `CONTEXT.md`, call it out immediately. "Your glossary defines 'cancellation' as X, but you seem to mean Y — which is it?"
### Sharpen fuzzy language
When the user uses vague or overloaded terms, propose a precise canonical term. "You're saying 'account' — do you mean the Customer or the User? Those are different things."
### Discuss concrete scenarios
When domain relationships are being discussed, stress-test them with specific scenarios. Invent scenarios that probe edge cases and force the user to be precise about the boundaries between concepts.
### Cross-reference with code
When the user states how something works, check whether the code agrees. If you find a contradiction, surface it: "Your code cancels entire Orders, but you just said partial cancellation is possible — which is right?"
### Update CONTEXT.md inline
When a term is resolved, update `CONTEXT.md` right there. Don't batch these up — capture them as they happen. Use the format in [CONTEXT-FORMAT.md](./CONTEXT-FORMAT.md).
`CONTEXT.md` should be totally devoid of implementation details. Do not treat `CONTEXT.md` as a spec, a scratch pad, or a repository for implementation decisions. It is a glossary and nothing else.
### Offer ADRs sparingly
Only offer to create an ADR when all three are true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will wonder "why did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If any of the three is missing, skip the ADR. Use the format in [ADR-FORMAT.md](./ADR-FORMAT.md).
</supporting-info>
Run a `/grilling` session, using the `/domain-modeling` skill.

View File

@@ -2,6 +2,7 @@
name: handoff
description: Compact the current conversation into a handoff document for another agent to pick up.
argument-hint: "What will the next session be used for?"
disable-model-invocation: true
---
Write a handoff document summarising the current conversation so a fresh agent can continue the work. Save to the temporary directory of the user's OS - not the current workspace.

View File

@@ -1,37 +0,0 @@
# Deepening
How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [LANGUAGE.md](LANGUAGE.md) — **module**, **interface**, **seam**, **adapter**.
## Dependency categories
When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam.
### 1. In-process
Pure computation, in-memory state, no I/O. Always deepenable — merge the modules and test through the new interface directly. No adapter needed.
### 2. Local-substitutable
Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface.
### 3. Remote but owned (Ports & Adapters)
Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter.
Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."*
### 4. True external (Mock)
Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter.
## Seam discipline
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection.
- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them.
## Testing strategy: replace, don't layer
- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist — delete them.
- Write new tests at the deepened module's interface. The **interface is the test surface**.
- Tests assert on observable outcomes through the interface, not internal state.
- Tests should survive internal refactors — they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface.

View File

@@ -39,7 +39,7 @@ Repo name, date, and a compact legend: solid box = module, dashed line = seam, r
## Candidate card
The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms ([LANGUAGE.md](LANGUAGE.md)) without ceremony.
The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms (from the `/codebase-design` skill) without ceremony.
Each candidate is one `<article>`:
@@ -105,7 +105,7 @@ One larger card. Candidate name, one sentence on why, anchor link to its card. T
## Tone
Plain English, concise — but the architectural nouns and verbs come straight from [LANGUAGE.md](LANGUAGE.md). Concision is not an excuse to drift.
Plain English, concise — but the architectural nouns and verbs come straight from the `/codebase-design` skill. Concision is not an excuse to drift.
**Use exactly:** module, interface, implementation, depth, deep, shallow, seam, adapter, leverage, locality.
@@ -120,4 +120,4 @@ Plain English, concise — but the architectural nouns and verbs come straight f
**Wins bullets** name the gain in glossary terms: *"locality: bugs concentrate in one module"*, *"leverage: one interface, N call sites"*, *"interface shrinks; implementation absorbs the wrappers"*. Don't write *"easier to maintain"* or *"cleaner code"* — those terms aren't in the glossary and don't earn their place.
No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in [LANGUAGE.md](LANGUAGE.md), reach for one that is before inventing a new one.
No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in the `/codebase-design` glossary, reach for one that is before inventing a new one.

View File

@@ -1,44 +0,0 @@
# Interface Design
When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout) — your first idea is unlikely to be the best.
Uses the vocabulary in [LANGUAGE.md](LANGUAGE.md) — **module**, **interface**, **seam**, **adapter**, **leverage**.
## Process
### 1. Frame the problem space
Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate:
- The constraints any new interface would need to satisfy
- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md))
- A rough illustrative code sketch to ground the constraints — not a proposal, just a way to make the constraints concrete
Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel.
### 2. Spawn sub-agents
Spawn 3+ sub-agents in parallel using the Agent tool. Each must produce a **radically different** interface for the deepened module.
Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint:
- Agent 1: "Minimize the interface — aim for 13 entry points max. Maximise leverage per entry point."
- Agent 2: "Maximise flexibility — support many use cases and extension."
- Agent 3: "Optimise for the most common caller — make the default case trivial."
- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies."
Include both [LANGUAGE.md](LANGUAGE.md) vocabulary and CONTEXT.md vocabulary in the brief so each sub-agent names things consistently with the architecture language and the project's domain language.
Each sub-agent outputs:
1. Interface (types, methods, params — plus invariants, ordering, error modes)
2. Usage example showing how callers use it
3. What the implementation hides behind the seam
4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md))
5. Trade-offs — where leverage is high, where it's thin
### 3. Present and compare
Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**.
After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated — the user wants a strong read, not a menu.

View File

@@ -1,53 +0,0 @@
# Language
Shared vocabulary for every suggestion this skill makes. Use these terms exactly — don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point.
## Terms
**Module**
Anything with an interface and an implementation. Deliberately scale-agnostic — applies equally to a function, class, package, or tier-spanning slice.
_Avoid_: unit, component, service.
**Interface**
Everything a caller must know to use the module correctly. Includes the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics.
_Avoid_: API, signature (too narrow — those refer only to the type-level surface).
**Implementation**
What's inside a module — its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise.
**Depth**
Leverage at the interface — the amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface. A module is **shallow** when the interface is nearly as complex as the implementation.
**Seam** _(from Michael Feathers)_
A place where you can alter behaviour without editing in that place. The *location* at which a module's interface lives. Choosing where to put the seam is its own design decision, distinct from what goes behind it.
_Avoid_: boundary (overloaded with DDD's bounded context).
**Adapter**
A concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside).
**Leverage**
What callers get from depth. More capability per unit of interface they have to learn. One implementation pays back across N call sites and M tests.
**Locality**
What maintainers get from depth. Change, bugs, knowledge, and verification concentrate at one place rather than spreading across callers. Fix once, fixed everywhere.
## Principles
- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts — they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface.
- **The deletion test.** Imagine deleting the module. If complexity vanishes, the module wasn't hiding anything (it was a pass-through). If complexity reappears across N callers, the module was earning its keep.
- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape.
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it.
## Relationships
- A **Module** has exactly one **Interface** (the surface it presents to callers and tests).
- **Depth** is a property of a **Module**, measured against its **Interface**.
- A **Seam** is where a **Module**'s **Interface** lives.
- An **Adapter** sits at a **Seam** and satisfies the **Interface**.
- **Depth** produces **Leverage** for callers and **Locality** for maintainers.
## Rejected framings
- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead.
- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow — interface here includes every fact a caller must know.
- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**.

View File

@@ -1,38 +1,23 @@
---
name: improve-codebase-architecture
description: Find deepening opportunities in a codebase, informed by the domain language in CONTEXT.md and the decisions in docs/adr/. Use when the user wants to improve architecture, find refactoring opportunities, consolidate tightly-coupled modules, or make a codebase more testable and AI-navigable.
description: Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.
disable-model-invocation: true
---
# Improve Codebase Architecture
Surface architectural friction and propose **deepening opportunities** — refactors that turn shallow modules into deep ones. The aim is testability and AI-navigability.
## Glossary
This command is _informed_ by the project's domain model and built on a shared design vocabulary:
Use these terms exactly in every suggestion. Consistent language is the point — don't drift into "component," "service," "API," or "boundary." Full definitions in [LANGUAGE.md](LANGUAGE.md).
- **Module** — anything with an interface and an implementation (function, class, package, slice).
- **Interface** — everything a caller must know to use the module: types, invariants, error modes, ordering, config. Not just the type signature.
- **Implementation** — the code inside.
- **Depth** — leverage at the interface: a lot of behaviour behind a small interface. **Deep** = high leverage. **Shallow** = interface nearly as complex as the implementation.
- **Seam** — where an interface lives; a place behaviour can be altered without editing in place. (Use this, not "boundary.")
- **Adapter** — a concrete thing satisfying an interface at a seam.
- **Leverage** — what callers get from depth.
- **Locality** — what maintainers get from depth: change, bugs, knowledge concentrated in one place.
Key principles (see [LANGUAGE.md](LANGUAGE.md) for the full list):
- **Deletion test**: imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep.
- **The interface is the test surface.**
- **One adapter = hypothetical seam. Two adapters = real seam.**
This skill is _informed_ by the project's domain model. The domain language gives names to good seams; ADRs record decisions the skill should not re-litigate.
- Run the `/codebase-design` skill for the architecture vocabulary (**module**, **interface**, **depth**, **seam**, **adapter**, **leverage**, **locality**) and its principles (the deletion test, "the interface is the test surface", "one adapter = hypothetical seam, two = real"). Use these terms exactly in every suggestion — don't drift into "component," "service," "API," or "boundary."
- The domain language in `CONTEXT.md` gives names to good seams; ADRs in `docs/adr/` record decisions this command should not re-litigate.
## Process
### 1. Explore
Read the project's domain glossary and any ADRs in the area you're touching first.
Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first.
Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
@@ -50,7 +35,7 @@ Write a self-contained HTML file to the OS temp directory so nothing lands in th
The report uses **Tailwind via CDN** for layout and styling, and **Mermaid via CDN** for diagrams where a graph/flow/sequence reliably communicates the structure. Mix Mermaid with hand-crafted CSS/SVG visuals — use Mermaid when relationships are graph-shaped (call graphs, dependencies, sequences), and hand-built divs/SVG when you want something more editorial (mass diagrams, cross-sections, collapse animations). Each candidate gets a **before/after visualisation**. Be visual.
For each candidate, the same template as before, but rendered as a card:
For each candidate, render a card with:
- **Files** — which files/modules are involved
- **Problem** — why the current architecture is causing friction
@@ -61,7 +46,7 @@ For each candidate, the same template as before, but rendered as a card:
End the report with a **Top recommendation** section: which candidate you'd tackle first and why.
**Use CONTEXT.md vocabulary for the domain, and [LANGUAGE.md](LANGUAGE.md) vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service."
**Use CONTEXT.md vocabulary for the domain, and the `/codebase-design` vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service."
**ADR conflicts**: if a candidate contradicts an existing ADR, only surface it when the friction is real enough to warrant revisiting the ADR. Mark it clearly in the card (e.g. a warning callout: _"contradicts ADR-0007 — but worth reopening because…"_). Don't list every theoretical refactor an ADR forbids.
@@ -71,11 +56,11 @@ Do NOT propose interfaces yet. After the file is written, ask the user: "Which o
### 3. Grilling loop
Once the user picks a candidate, drop into a grilling conversation. Walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Once the user picks a candidate, run the `/grilling` skill to walk the design tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Side effects happen inline as decisions crystallize:
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md` — same discipline as `/grill-with-docs` (see [CONTEXT-FORMAT.md](../grill-with-docs/CONTEXT-FORMAT.md)). Create the file lazily if it doesn't exist.
- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md`. Create the file lazily if it doesn't exist.
- **Sharpening a fuzzy term during the conversation?** Update `CONTEXT.md` right there.
- **User rejects the candidate with a load-bearing reason?** Offer an ADR, framed as: _"Want me to record this as an ADR so future architecture reviews don't re-suggest it?"_ Only offer when the reason would actually be needed by a future explorer to avoid re-suggesting the same thing — skip ephemeral reasons ("not worth it right now") and self-evident ones. See [ADR-FORMAT.md](../grill-with-docs/ADR-FORMAT.md).
- **Want to explore alternative interfaces for the deepened module?** See [INTERFACE-DESIGN.md](INTERFACE-DESIGN.md).
- **User rejects the candidate with a load-bearing reason?** Offer an ADR, framed as: _"Want me to record this as an ADR so future architecture reviews don't re-suggest it?"_ Only offer when the reason would actually be needed by a future explorer to avoid re-suggesting the same thing — skip ephemeral reasons ("not worth it right now") and self-evident ones.
- **Want to explore alternative interfaces for the deepened module?** Run the `/codebase-design` skill and use its design-it-twice parallel sub-agent pattern.

View File

@@ -1,6 +1,6 @@
---
name: prototype
description: Build a throwaway prototype to flesh out a design before committing to it. Routes between two branches — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route. Use when the user wants to prototype, sanity-check a data model or state machine, mock up a UI, explore design options, or says "prototype this", "let me play with it", "try a few designs".
description: Build a throwaway prototype to answer a design question. Use when the user wants to sanity-check whether a state model or logic feels right, or explore what a UI should look like.
---
# Prototype

View File

@@ -1,6 +1,6 @@
---
name: setup-matt-pocock-skills
description: Sets up an `## Agent skills` block in AGENTS.md/CLAUDE.md and `docs/agents/` so the engineering skills know this repo's issue tracker (GitHub or local markdown), triage label vocabulary, and domain doc layout. Run before first use of `to-issues`, `to-prd`, `triage`, `diagnose`, `tdd`, `improve-codebase-architecture`, or `zoom-out` — or if those skills appear to be missing context about the issue tracker, triage labels, or domain docs.
description: Configure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.
disable-model-invocation: true
---
@@ -44,6 +44,12 @@ Default posture: these skills were designed for GitHub. If a `git remote` points
- **Local markdown** — issues live as files under `.scratch/<feature>/` in this repo (good for solo projects or repos without a remote)
- **Other** (Jira, Linear, etc.) — ask the user to describe the workflow in one paragraph; the skill will record it as freeform prose
If — and only if — the user picked **GitHub** or **GitLab**, ask one follow-up:
> Explainer: Open-source repos often receive feature requests as pull requests, not just issues — a PR is an issue with attached code. If you turn this on, `/triage` pulls *external* PRs into the same queue and runs them through the same labels and states as issues (collaborators' in-flight PRs are left alone). Leave it off if PRs aren't a request surface for you.
- **PRs as a request surface** — yes / no (default: no). Record the answer in `docs/agents/issue-tracker.md`. For local-markdown and other trackers, skip this question — there are no PRs.
**Section B — Triage label vocabulary.**
> Explainer: When the `triage` skill processes an incoming issue, it moves it through a state machine — needs evaluation, waiting on reporter, ready for an AFK agent to pick up, ready for a human, or won't fix. To do that, it needs to apply labels (or the equivalent in your issue tracker) that match strings *you've actually configured*. If your repo already uses different label names (e.g. `bug:triage` instead of `needs-triage`), map them here so the skill applies the right ones instead of creating duplicates.
@@ -60,7 +66,7 @@ Default: each role's string equals its name. Ask the user if they want to overri
**Section C — Domain docs.**
> Explainer: Some skills (`improve-codebase-architecture`, `diagnose`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
> Explainer: Some skills (`improve-codebase-architecture`, `diagnosing-bugs`, `tdd`) read a `CONTEXT.md` file to learn the project's domain language, and `docs/adr/` for past architectural decisions. They need to know whether the repo has one global context or multiple (e.g. a monorepo with separate frontend/backend contexts) so they look in the right place.
Confirm the layout:
@@ -95,7 +101,7 @@ The block:
### Issue tracker
[one-line summary of where issues are tracked]. See `docs/agents/issue-tracker.md`.
[one-line summary of where issues are tracked, plus whether external PRs are a triage surface]. See `docs/agents/issue-tracker.md`.
### Triage labels

View File

@@ -8,7 +8,7 @@ How the engineering skills should consume this repo's domain documentation when
- **`CONTEXT-MAP.md`** at the repo root if it exists — it points at one `CONTEXT.md` per context. Read each one relevant to the topic.
- **`docs/adr/`** — read ADRs that touch the area you're about to work in. In multi-context repos, also check `src/<context>/docs/adr/` for context-scoped decisions.
If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The producer skill (`/grill-with-docs`) creates them lazily when terms or decisions actually get resolved.
If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill (reached via `/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms or decisions actually get resolved.
## File structure
@@ -42,7 +42,7 @@ Multi-context repo (presence of `CONTEXT-MAP.md` at the root):
When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids.
If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/grill-with-docs`).
If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`).
## Flag ADR conflicts

View File

@@ -13,6 +13,18 @@ Issues and PRDs for this repo live as GitHub issues. Use the `gh` CLI for all op
Infer the repo from `git remote -v``gh` does this automatically when run inside a clone.
## Pull requests as a triage surface
**PRs as a request surface: no.** _(Set to `yes` if this repo treats external PRs as feature requests; `/triage` reads this flag.)_
When set to `yes`, PRs run through the same labels and states as issues, using the `gh pr` equivalents:
- **Read a PR**: `gh pr view <number> --comments` and `gh pr diff <number>` for the diff.
- **List external PRs for triage**: `gh pr list --state open --json number,title,body,labels,author,authorAssociation,comments` then keep only `authorAssociation` of `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR`, or `NONE` (drop `OWNER`/`MEMBER`/`COLLABORATOR`).
- **Comment / label / close**: `gh pr comment`, `gh pr edit --add-label`/`--remove-label`, `gh pr close`.
GitHub shares one number space across issues and PRs, so a bare `#42` may be either — resolve with `gh pr view 42` and fall back to `gh issue view 42`.
## When a skill says "publish to the issue tracker"
Create a GitHub issue.

View File

@@ -14,6 +14,18 @@ Issues and PRDs for this repo live as GitLab issues. Use the [`glab`](https://gi
Infer the repo from `git remote -v``glab` does this automatically when run inside a clone.
## Merge requests as a triage surface
**MRs as a request surface: no.** _(Set to `yes` if this repo treats external merge requests as feature requests; `/triage` reads this flag.)_
When set to `yes`, MRs run through the same labels and states as issues, using the `glab mr` equivalents:
- **Read an MR**: `glab mr view <number> --comments` and `glab mr diff <number>` for the diff.
- **List external MRs for triage**: `glab mr list -F json`, then keep only MRs whose author is not a project member/owner (a contributor's MR, not a maintainer's in-flight work).
- **Comment / label / close**: `glab mr note`, `glab mr update --label`/`--unlabel`, `glab mr close`.
Unlike GitHub, GitLab numbers issues and MRs separately, so `#42` is unambiguous once you know which surface the maintainer means.
## When a skill says "publish to the issue tracker"
Create a GitLab issue.

View File

@@ -1,6 +1,6 @@
---
name: tdd
description: Test-driven development with red-green-refactor loop. Use when user wants to build features or fix bugs using TDD, mentions "red-green-refactor", wants integration tests, or asks for test-first development.
description: Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
---
# Test-Driven Development
@@ -13,6 +13,8 @@ description: Test-driven development with red-green-refactor loop. Use when user
**Bad tests** are coupled to implementation. They mock internal collaborators, test private methods, or verify through external means (like querying a database directly instead of using the interface). The warning sign: your test breaks when you refactor, but behavior hasn't changed. If you rename an internal function and tests fail, those tests were testing implementation, not behavior.
**Tautological tests** restate the implementation inside the assertion, so they pass by construction and give zero confidence. When the expected value is computed the way the code computes it — `expect(add(a, b)).toBe(a + b)`, snapshotting a figure you derived by hand the same way the code does, asserting a constant equals itself — the test can never disagree with the code: break the code wrong and the assertion breaks wrong with it. The expected value must come from an independent source of truth — a known-good literal, a worked example, the spec.
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
## Anti-Pattern: Horizontal Slices
@@ -44,14 +46,13 @@ RIGHT (vertical):
### 1. Planning
When exploring the codebase, use the project's domain glossary so that test names and interface vocabulary match the project's language, and respect ADRs in the area you're touching.
When exploring the codebase, read `CONTEXT.md` (if it exists) so that test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
Before writing any code:
- [ ] Confirm with user what interface changes are needed
- [ ] Confirm with user which behaviors to test (prioritize)
- [ ] Identify opportunities for [deep modules](deep-modules.md) (small interface, deep implementation)
- [ ] Design interfaces for [testability](interface-design.md)
- [ ] Identify opportunities for deep modules (small interface, deep implementation) — run the `/codebase-design` skill for the vocabulary and the testability checks
- [ ] List the behaviors to test (not implementation steps)
- [ ] Get user approval on the plan
@@ -104,6 +105,7 @@ After all tests pass, look for [refactor candidates](refactoring.md):
[ ] Test describes behavior, not implementation
[ ] Test uses public interface only
[ ] Test would survive internal refactor
[ ] Expected values are independent literals, not recomputed from the code
[ ] Code is minimal for this test
[ ] No speculative features added
```

View File

@@ -1,33 +0,0 @@
# Deep Modules
From "A Philosophy of Software Design":
**Deep module** = small interface + lots of implementation
```
┌─────────────────────┐
│ Small Interface │ ← Few methods, simple params
├─────────────────────┤
│ │
│ │
│ Deep Implementation│ ← Complex logic hidden
│ │
│ │
└─────────────────────┘
```
**Shallow module** = large interface + little implementation (avoid)
```
┌─────────────────────────────────┐
│ Large Interface │ ← Many methods, complex params
├─────────────────────────────────┤
│ Thin Implementation │ ← Just passes through
└─────────────────────────────────┘
```
When designing interfaces, ask:
- Can I reduce the number of methods?
- Can I simplify the parameters?
- Can I hide more complexity inside?

View File

@@ -1,31 +0,0 @@
# Interface Design for Testability
Good interfaces make testing natural:
1. **Accept dependencies, don't create them**
```typescript
// Testable
function processOrder(order, paymentGateway) {}
// Hard to test
function processOrder(order) {
const gateway = new StripeGateway();
}
```
2. **Return results, don't produce side effects**
```typescript
// Testable
function calculateDiscount(cart): Discount {}
// Hard to test
function applyDiscount(cart): void {
cart.total -= discount;
}
```
3. **Small surface area**
- Fewer methods = fewer tests needed
- Fewer params = simpler test setup

View File

@@ -59,3 +59,19 @@ test("createUser makes user retrievable", async () => {
expect(retrieved.name).toBe("Alice");
});
```
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
```typescript
// BAD: Expected value is recomputed the way the code computes it
test("calculateTotal sums line items", () => {
const items = [{ price: 10 }, { price: 5 }];
const expected = items.reduce((sum, i) => sum + i.price, 0);
expect(calculateTotal(items)).toBe(expected);
});
// GOOD: Expected value is an independent, known literal
test("calculateTotal sums line items", () => {
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
});
```

View File

@@ -16,6 +16,7 @@ Treat the current directory as a teaching workspace. The state of their learning
- `RESOURCES.md`: A list of resources which can be explored to ground your teaching in contextual knowledge, or to acquire knowledge and wisdom. Use the format in [RESOURCES-FORMAT.md](./RESOURCES-FORMAT.md).
- `./learning-records/*.md`: A directory of learning records, which capture what the user has learned. These are loosely equivalent to architectural decision records in software development - they capture non-obvious lessons and key insights that may need to be revised later, or drive future sessions. These should be used to calculate the zone of proximal development. They are titled `0001-<dash-case-name>.md`, where the number increments each time. Use the format in [LEARNING-RECORD-FORMAT.md](./LEARNING-RECORD-FORMAT.md).
- `./lessons/*.html`: A directory of lessons. A **lesson** is a single, self-contained HTML output that teaches one tightly-scoped thing tied to the mission. This is the primary unit of teaching in this workspace.
- `./assets/*`: Reusable **components** shared across lessons. See [Assets](#assets).
- `NOTES.md`: A scratchpad for you to jot down user preferences, or working notes.
## Philosophy
@@ -59,6 +60,14 @@ Each lesson should recommend a primary source for the user to read or watch. Thi
Each lesson should contain a reminder to ask followup questions to the agent. The agent is their teacher, and can assist with anything that's unclear.
## Assets
Lessons are built from reusable **components**, stored in `./assets/`: stylesheets, quiz widgets, simulators, diagram helpers — anything a second lesson could reuse.
Reuse is the default, not the exception. Before authoring a lesson, read `./assets/` and build from the components already there. When a lesson needs something new and reusable, write it as a component in `./assets/` and link to it — never inline code a future lesson would duplicate.
A shared stylesheet is the first component every workspace earns: every lesson links it, so the lessons look like one consistent course rather than a pile of one-offs. As the workspace grows, so should the component library.
## The Mission
Every lesson should be tied into the mission - the reason that the user is interested in learning about the topic.

View File

@@ -1,6 +1,7 @@
---
name: to-issues
description: Break a plan, spec, or PRD into independently-grabbable issues on the project issue tracker using tracer-bullet vertical slices. Use when user wants to convert a plan into issues, create implementation tickets, or break down work into issues.
description: Break a plan, spec, or PRD into independently-grabbable issues on the project issue tracker using tracer-bullet vertical slices.
disable-model-invocation: true
---
# To Issues
@@ -19,16 +20,18 @@ Work from whatever is already in the conversation context. If the user passes an
If you have not already explored the codebase, do so to understand the current state of the code. Issue titles and descriptions should use the project's domain glossary vocabulary, and respect ADRs in the area you're touching.
Look for opportunities to prefactor the code to make the implementation easier. "Make the change easy, then make the easy change."
### 3. Draft vertical slices
Break the plan into **tracer bullet** issues. Each issue is a thin vertical slice that cuts through ALL integration layers end-to-end, NOT a horizontal slice of one layer.
Slices may be 'HITL' or 'AFK'. HITL slices require human interaction, such as an architectural decision or a design review. AFK slices can be implemented and merged without human interaction. Prefer AFK over HITL where possible.
<vertical-slice-rules>
- Each slice delivers a narrow but COMPLETE path through every layer (schema, API, UI, tests)
- A completed slice is demoable or verifiable on its own
- Prefer many thin slices over few thick ones
- Any prefactoring should be done first
</vertical-slice-rules>
### 4. Quiz the user
@@ -36,7 +39,6 @@ Slices may be 'HITL' or 'AFK'. HITL slices require human interaction, such as an
Present the proposed breakdown as a numbered list. For each slice, show:
- **Title**: short descriptive name
- **Type**: HITL / AFK
- **Blocked by**: which other slices (if any) must complete first
- **User stories covered**: which user stories this addresses (if the source material has them)
@@ -45,7 +47,6 @@ Ask the user:
- Does the granularity feel right? (too coarse / too fine)
- Are the dependency relationships correct?
- Should any slices be merged or split further?
- Are the correct slices marked as HITL and AFK?
Iterate until the user approves the breakdown.

View File

@@ -1,6 +1,7 @@
---
name: to-prd
description: Turn the current conversation context into a PRD and publish it to the project issue tracker. Use when user wants to create a PRD from the current context.
description: Turn the current conversation into a PRD and publish it to the project issue tracker — no interview, just synthesis of what you've already discussed.
disable-model-invocation: true
---
This skill takes the current conversation context and codebase understanding and produces a PRD. Do NOT interview the user — just synthesize what you already know.
@@ -11,7 +12,7 @@ The issue tracker and triage label vocabulary should have been provided to you
1. Explore the repo to understand the current state of the codebase, if you haven't already. Use the project's domain glossary vocabulary throughout the PRD, and respect any ADRs in the area you're touching.
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can.
2. Sketch out the seams at which you're going to test the feature. Existing seams should be preferred to new ones. Use the highest seam possible. If new seams are needed, propose them at the highest point you can. The fewer seams across the codebase, the better - the ideal number is one.
Check with the user that these seams match their expectations.

View File

@@ -1,6 +1,8 @@
# Writing Agent Briefs
An agent brief is a structured comment posted on a GitHub issue when it moves to `ready-for-agent`. It is the authoritative specification that an AFK agent will work from. The original issue body and discussion are context — the agent brief is the contract.
An agent brief is a structured comment posted on a GitHub issue or PR when it moves to `ready-for-agent`. It is the authoritative specification that an AFK agent will work from. The original body and discussion are context — the agent brief is the contract.
The brief states **what the agent should do**, which stretches to both surfaces: for an issue, that's building the change from nothing; for a PR, it's what's left to do *to the existing diff* — finish it, close gaps, address review points. Same principles either way; the PR example below shows the difference.
## Principles
@@ -143,6 +145,43 @@ checked for matches.
- Bug reports (only enhancement rejections go to `.out-of-scope/`)
```
### Good agent brief (PR)
For a PR, "Current behavior" describes the state of the diff, and the brief asks the agent to finish or fix it rather than build from scratch.
```markdown
## Agent Brief
**Category:** enhancement
**Summary:** Finish the contributor's `--json` output flag for `triage list`
**Current behavior:**
The PR adds a `--json` flag that serializes the issue list to JSON. The happy
path works and the diff matches the project's command structure. Two gaps
remain: errors are still printed as human text (not JSON), and the new flag has
no test coverage.
**Desired behavior:**
With `--json`, all output — including errors — is well-formed JSON on stdout,
and the command's exit codes are unchanged. The existing human-readable output
is untouched when the flag is absent.
**Key interfaces:**
- The command's error path should emit `{ "error": string }` under `--json`
instead of the plain-text error
- Reuse the existing serializer the PR already added; don't introduce a second
**Acceptance criteria:**
- [ ] `triage list --json` emits valid JSON for both success and error cases
- [ ] Exit codes match the non-JSON command
- [ ] A test covers the `--json` success output and one error case
- [ ] Default (non-JSON) output is byte-for-byte unchanged
**Out of scope:**
- Adding `--json` to any other command
- Changing the JSON shape of the success payload the PR already defined
```
### Bad agent brief
```markdown

View File

@@ -83,7 +83,11 @@ The maintainer may:
## When to write to `.out-of-scope/`
Only when an **enhancement** (not a bug) is rejected as `wontfix`. The flow:
Only when an **enhancement** (not a bug) is *rejected* as `wontfix`. This applies to enhancement PRs exactly as it does to issues — a rejected PR is recorded here so the same request doesn't return as fresh code.
Do **not** write here when something is closed as `wontfix` because it's **already implemented**. That's a built feature, not a rejected one; recording it would poison the dedup checks with false rejections. Instead, the closing comment points to where the feature already lives.
The flow:
1. Maintainer decides a feature request is out of scope
2. Check if a matching `.out-of-scope/` file already exists

View File

@@ -1,12 +1,15 @@
---
name: triage
description: Triage issues through a state machine driven by triage roles. Use when user wants to create an issue, triage issues, review incoming bugs or feature requests, prepare issues for an AFK agent, or manage issue workflow.
description: Move issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.
disable-model-invocation: true
---
# Triage
Move issues on the project issue tracker through a small state machine of triage roles.
If this repo treats external pull requests as a request surface (see the issue-tracker config), triage covers them too: **a PR is an issue with attached code** — same roles, same states, same machine, with a few deltas marked "for a PR" below. Resolve a bare `#42` to an issue or PR per the tracker config.
Every comment or issue posted to the issue tracker during triage **must** start with this disclaimer:
```
@@ -33,6 +36,8 @@ Five **state** roles:
- `ready-for-human` — needs human implementation
- `wontfix` — will not be actioned
For a PR, the same states read against the attached code: `ready-for-agent` means a brief is attached and an agent should take the next step on the diff; `ready-for-human` means it's ready for a human to merge.
Every triaged issue should carry exactly one category role and one state role. If state roles conflict, flag it and ask the maintainer before doing anything else.
These are canonical role names — the actual label strings used in the issue tracker may differ. The mapping should have been provided to you - run `/setup-matt-pocock-skills` if not.
@@ -44,7 +49,7 @@ State transitions: an unlabeled issue normally goes to `needs-triage` first; fro
The maintainer invokes `/triage` and describes what they want in natural language. Interpret the request and act. Examples:
- "Show me anything that needs my attention"
- "Let's look at #42"
- "Let's look at #42" (issue or PR)
- "Move #42 to ready-for-agent"
- "What's ready for agents to pick up?"
@@ -56,24 +61,28 @@ Query the issue tracker and present three buckets, oldest first:
2. **`needs-triage`** — evaluation in progress.
3. **`needs-info` with reporter activity since the last triage notes** — needs re-evaluation.
Show counts and a one-line summary per issue. Let the maintainer pick.
When PRs are in scope, include external PRs in these buckets and tag each line `[PR]` or `[issue]`. Discovery surfaces only *external* PRs (the tracker config defines who counts as external) — a collaborator's in-flight PR is not triage work. This filter is discovery-only; an explicitly named PR is always triaged regardless of author.
## Triage a specific issue
Show counts and a one-line summary per item. Let the maintainer pick.
1. **Gather context.** Read the full issue (body, comments, labels, reporter, dates). Parse any prior triage notes so you don't re-ask resolved questions. Explore the codebase using the project's domain glossary, respecting ADRs in the area. Read `.out-of-scope/*.md` and surface any prior rejection that resembles this issue.
## Triage a specific issue or PR
2. **Recommend.** Tell the maintainer your category and state recommendation with reasoning, plus a brief codebase summary relevant to the issue. Wait for direction.
1. **Gather context.** Read the full issue or PR (body, comments, labels, author, dates; for a PR, the diff too). Parse any prior triage notes so you don't re-ask resolved questions. Explore the codebase using the project's domain glossary, respecting ADRs in the area. Run two checks against the codebase: (a) **redundancy** — search for an existing implementation of the requested behavior by domain concept (not just the request's wording), and report where you looked. If found, it's an already-implemented `wontfix` (step 5). (b) **prior rejection** — read `.out-of-scope/*.md` and surface any that resembles this request.
3. **Reproduce (bugs only).** Before any grilling, attempt reproduction: read the reporter's steps, trace the relevant code, run tests or commands. Report what happened — successful repro with code path, failed repro, or insufficient detail (a strong `needs-info` signal). A confirmed repro makes a much stronger agent brief.
2. **Recommend.** Tell the maintainer your category and state recommendation with reasoning, plus a brief codebase summary relevant to the request — including whether it's already implemented. Wait for direction.
4. **Grill (if needed).** If the issue needs fleshing out, run a `/grill-with-docs` session.
3. **Verify the claim.** Before any grilling, check that the claim holds up. For a bug, reproduce it from the reporter's steps. For a PR, confirm the diff does what it claims — check it out, run the relevant tests or commands. Report what happened: confirmed (with code path), failed, or insufficient detail (a strong `needs-info` signal). A confirmed verification makes a much stronger agent brief.
4. **Grill (if needed).** If the request needs fleshing out, run the `/grilling` and `/domain-modeling` skills together — grill it into shape one question at a time, sharpening domain terms and updating `CONTEXT.md`/ADRs inline as decisions land.
5. **Apply the outcome:**
- `ready-for-agent` — post an agent brief comment ([AGENT-BRIEF.md](AGENT-BRIEF.md)).
- `ready-for-human` — same structure as an agent brief, but note why it can't be delegated (judgment calls, external access, design decisions, manual testing).
- `needs-info` — post triage notes (template below).
- `wontfix` (bug) — polite explanation, then close.
- `wontfix` (enhancement) — write to `.out-of-scope/`, link to it from a comment, then close ([OUT-OF-SCOPE.md](OUT-OF-SCOPE.md)).
- `wontfix` — close, with the comment depending on *why*:
- **Already implemented** — the change already exists in the codebase. Point to where it lives; do **not** write to `.out-of-scope/` (that KB is for *rejected* requests, not built ones).
- **Rejected (bug)** — polite explanation, then close.
- **Rejected (enhancement)** — write to `.out-of-scope/`, link to it from a comment, then close ([OUT-OF-SCOPE.md](OUT-OF-SCOPE.md)).
- `needs-triage` — apply the role. Optional comment if there's partial progress.
## Quick state override
@@ -100,4 +109,4 @@ Capture everything resolved during grilling under "established so far" so the wo
## Resuming a previous session
If prior triage notes exist on the issue, read them, check whether the reporter has answered any outstanding questions, and present an updated picture before continuing. Don't re-ask resolved questions.
If prior triage notes exist on the issue or PR, read them, check whether the reporter has answered any outstanding questions, and present an updated picture before continuing. Don't re-ask resolved questions.

View File

@@ -56,12 +56,13 @@ func (h *AssetHandler) SetLifecycleService(svc AssetLifecycleService) {
}
// resolveByIdentifier 通过标识符解析资产,用于各处理器的公共逻辑
// 不传递 include_usage_summary始终为 false不影响其他 Handler 性能
func (h *AssetHandler) resolveByIdentifier(c *fiber.Ctx) (*dto.AssetResolveResponse, error) {
identifier := c.Params("identifier")
if identifier == "" {
return nil, errors.New(errors.CodeInvalidParam, "标识符不能为空")
}
return h.assetService.Resolve(c.UserContext(), identifier)
return h.assetService.Resolve(c.UserContext(), identifier, false)
}
// Resolve 通过任意标识符解析资产(设备或卡)
@@ -72,7 +73,8 @@ func (h *AssetHandler) Resolve(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "标识符不能为空")
}
result, err := h.assetService.Resolve(c.UserContext(), identifier)
includeUsageSummary := c.QueryBool("include_usage_summary", false)
result, err := h.assetService.Resolve(c.UserContext(), identifier, includeUsageSummary)
if err != nil {
return err
}
@@ -378,7 +380,7 @@ func (h *AssetHandler) UpdatePollingStatus(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam)
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -424,7 +426,7 @@ func (h *AssetHandler) UpdateRealnamePolicy(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "无效的实名认证策略")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -467,7 +469,7 @@ func (h *AssetHandler) UpdateRealnameStatus(c *fiber.Ctx) error {
return errors.New(errors.CodeInvalidParam, "无效的实名状态")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}

View File

@@ -46,7 +46,7 @@ func (h *AssetWalletHandler) GetWallet(c *fiber.Ctx) error {
return errors.New(errors.CodeInternalError, "资产服务未配置")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}
@@ -76,7 +76,7 @@ func (h *AssetWalletHandler) ListTransactions(c *fiber.Ctx) error {
return errors.New(errors.CodeInternalError, "资产服务未配置")
}
asset, err := h.assetService.Resolve(c.UserContext(), identifier)
asset, err := h.assetService.Resolve(c.UserContext(), identifier, false)
if err != nil {
return err
}

View File

@@ -98,7 +98,7 @@ func (h *ClientAssetHandler) resolveAssetFromIdentifier(c *fiber.Ctx, identifier
}
skipPermissionCtx := context.WithValue(c.UserContext(), constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier)
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -69,7 +69,7 @@ func (h *ClientDeviceHandler) validateDeviceAsset(c *fiber.Ctx, identifier strin
ctx := c.UserContext()
// 通过标识符解析资产
asset, err := h.assetService.Resolve(ctx, identifier)
asset, err := h.assetService.Resolve(ctx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -84,7 +84,7 @@ func (h *ClientRealnameHandler) GetRealnameLink(c *fiber.Ctx) error {
ctx := c.UserContext()
// 3. 通过标识符解析资产
asset, err := h.assetService.Resolve(ctx, req.Identifier)
asset, err := h.assetService.Resolve(ctx, req.Identifier, false)
if err != nil {
return err
}

View File

@@ -279,6 +279,18 @@ func (h *ClientWalletHandler) CreateRecharge(c *fiber.Ctx) error {
return err
}
// 第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// switch resolved.Asset.AssetType {
// case "card":
// if req.PaymentMethod != constants.RechargeMethodAlipay {
// return errors.New(errors.CodeInvalidParam, "单卡资产只支持支付宝充值")
// }
// case "device":
// if req.PaymentMethod != constants.RechargeMethodWechat {
// return errors.New(errors.CodeInvalidParam, "设备资产只支持微信充值")
// }
// }
config, err := h.wechatConfigService.GetActiveConfig(resolved.SkipPermissionCtx)
if err != nil {
return err
@@ -540,7 +552,7 @@ func (h *ClientWalletHandler) resolveAssetFromIdentifier(c *fiber.Ctx, identifie
}
skipPermissionCtx := context.WithValue(c.UserContext(), constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier)
assetInfo, err := h.assetService.Resolve(skipPermissionCtx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -68,6 +68,9 @@ type AssetResolveResponse struct {
NetworkStatus int `json:"network_status" description:"网络状态0停机 1开机asset_type=card时有效"`
GatewayExtend string `json:"gateway_extend" description:"Gateway 卡状态扩展字段,原样返回上游 extend用于展示运营商侧实际停机原因"`
GatewayCardIMEI string `json:"gateway_card_imei" description:"插拔卡业务 IMEI由 Gateway 卡状态接口同步,非设备自身 IMEI无业务含义仅供查看"`
// 流量汇总字段(仅在 ?include_usage_summary=true 时返回非 null 值)
TotalVirtualUsedMB *float64 `json:"total_virtual_used_mb" description:"当前世代所有套餐虚已用之和MB未请求时为 null"`
TotalVirtualRemainingMB *float64 `json:"total_virtual_remaining_mb" description:"当前世代所有套餐虚剩余之和MB未请求时为 null"`
}
// BoundCardInfo 设备绑定的卡信息
@@ -148,9 +151,10 @@ type AssetPackagesResult struct {
Items []*AssetPackageResponse `json:"items" description:"套餐列表"`
}
// AssetResolveRequest 资产解析请求(路径参数)
// AssetResolveRequest 资产解析请求
type AssetResolveRequest struct {
Identifier string `path:"identifier" description:"资产标识符(虚拟号/ICCID/IMEI/SN/MSISDN" required:"true"`
Identifier string `path:"identifier" description:"资产标识符(虚拟号/ICCID/IMEI/SN/MSISDN" required:"true"`
IncludeUsageSummary bool `query:"include_usage_summary" description:"是否返回当前世代流量汇总字段total_virtual_used_mb / total_virtual_remaining_mb默认 false"`
}
// AssetIdentifierRequest 资产标识符路径参数请求

View File

@@ -484,7 +484,7 @@ func (s *Service) resolveOpenAPICard(ctx context.Context, cardNo string) (*model
return nil, errors.New(errors.CodeInternalError, "开放接口资产解析依赖未配置")
}
resolved, err := s.assetService.Resolve(ctx, identifier)
resolved, err := s.assetService.Resolve(ctx, identifier, false)
if err != nil {
if appErr, ok := err.(*errors.AppError); ok && appErr.Code == errors.CodeNotFound {
return s.resolveOpenAPICardByIdentifier(ctx, identifier)

View File

@@ -91,7 +91,8 @@ func New(
// Resolve 通过任意标识符解析资产
// 主路径:查注册表(精确匹配 ICCID 或 VirtualNo
// Fallback原有跨表 OR 查询(处理 IMEI/SN/MSISDN 等非注册标识符)
func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetResolveResponse, error) {
// includeUsageSummary 为 true 时额外填充当前世代流量汇总字段
func (s *Service) Resolve(ctx context.Context, identifier string, includeUsageSummary bool) (*dto.AssetResolveResponse, error) {
if s.assetIdentifierStore != nil {
regRecord, regErr := s.assetIdentifierStore.FindByIdentifier(ctx, identifier)
if regErr == nil && regRecord != nil {
@@ -102,6 +103,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildDeviceResolveResponse(ctx, device)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "device", device.ID)
}
}
return resp, buildErr
}
@@ -111,6 +115,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildCardResolveResponse(ctx, card)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "iot_card", card.ID)
}
}
return resp, buildErr
}
@@ -123,6 +130,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildDeviceResolveResponse(ctx, device)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "device", device.ID)
}
}
return resp, buildErr
}
@@ -135,6 +145,9 @@ func (s *Service) Resolve(ctx context.Context, identifier string) (*dto.AssetRes
resp, buildErr := s.buildCardResolveResponse(ctx, &card)
if buildErr == nil {
resp.Identifier = identifier
if includeUsageSummary {
s.fillUsageSummary(ctx, resp, "iot_card", card.ID)
}
}
return resp, buildErr
}
@@ -336,6 +349,19 @@ func (s *Service) fillPackageInfo(ctx context.Context, resp *dto.AssetResolveRes
resp.EnableVirtualData = usage.EnableVirtualDataSnapshot
}
// fillUsageSummary 填充当前世代流量汇总字段。
// 仅在 includeUsageSummary=true 时被调用,无套餐时返回 0.0(非 null
func (s *Service) fillUsageSummary(ctx context.Context, resp *dto.AssetResolveResponse, carrierType string, carrierID uint) {
usages, err := s.packageUsageStore.GetCurrentGenerationUsagesForSummary(ctx, carrierType, carrierID)
if err != nil {
logger.GetAppLogger().Error("查询流量汇总失败", zap.String("carrierType", carrierType), zap.Uint("carrierID", carrierID), zap.Error(err))
return
}
totalUsed, totalRemaining := computeUsageSummary(usages)
resp.TotalVirtualUsedMB = &totalUsed
resp.TotalVirtualRemainingMB = &totalRemaining
}
// fillShopName 填充店铺名称
func (s *Service) fillShopName(ctx context.Context, resp *dto.AssetResolveResponse) {
if resp.ShopID == nil || *resp.ShopID == 0 {
@@ -971,7 +997,7 @@ func (s *Service) GetOrders(ctx context.Context, identifier string, page, pageSi
pageSize = 100
}
asset, err := s.Resolve(ctx, identifier)
asset, err := s.Resolve(ctx, identifier, false)
if err != nil {
return nil, err
}

View File

@@ -0,0 +1,16 @@
package asset
import "github.com/break/junhong_cmp_fiber/internal/model"
// computeUsageSummary 汇总当前世代套餐的虚已用量和虚剩余量。
// 对每条记录调用 BuildTrafficMetrics() 获取虚流量字段,累加后计算剩余。
func computeUsageSummary(usages []*model.PackageUsage) (totalUsedMB float64, totalRemainingMB float64) {
var totalVirtualTotal float64
for _, u := range usages {
m := u.BuildTrafficMetrics()
totalUsedMB += m.VirtualUsedMB
totalVirtualTotal += float64(m.VirtualTotalMB)
}
totalRemainingMB = totalVirtualTotal - totalUsedMB
return
}

View File

@@ -0,0 +1,81 @@
package asset
import (
"testing"
"github.com/break/junhong_cmp_fiber/internal/model"
)
func TestComputeUsageSummary_EmptyList(t *testing.T) {
totalUsed, totalRemaining := computeUsageSummary(nil)
if totalUsed != 0.0 {
t.Errorf("期望 totalUsed=0.0,实际=%v", totalUsed)
}
if totalRemaining != 0.0 {
t.Errorf("期望 totalRemaining=0.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_SingleVirtualPackage(t *testing.T) {
// 启用虚流量:真实总量=100MB虚拟总量=200MB倍率=0.5
// 真实已用=40MB → 虚拟已用=min(40*0.5, 100)=20MB
// 虚拟剩余=200-20=180MB
usage := &model.PackageUsage{
DataLimitMB: 100,
DataUsageMB: 40,
VirtualTotalMBSnapshot: 200,
DisplayGainRatioSnapshot: 0.5,
EnableVirtualDataSnapshot: true,
}
totalUsed, totalRemaining := computeUsageSummary([]*model.PackageUsage{usage})
if totalUsed != 20.0 {
t.Errorf("期望 totalUsed=20.0,实际=%v", totalUsed)
}
if totalRemaining != 180.0 {
t.Errorf("期望 totalRemaining=180.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_SingleNonVirtualPackage(t *testing.T) {
// 未启用虚流量:退化为真实值
// 真实总量=100MB真实已用=40MB → 虚拟已用=40MB虚拟剩余=60MB
usage := &model.PackageUsage{
DataLimitMB: 100,
DataUsageMB: 40,
EnableVirtualDataSnapshot: false,
}
totalUsed, totalRemaining := computeUsageSummary([]*model.PackageUsage{usage})
if totalUsed != 40.0 {
t.Errorf("期望 totalUsed=40.0,实际=%v", totalUsed)
}
if totalRemaining != 60.0 {
t.Errorf("期望 totalRemaining=60.0,实际=%v", totalRemaining)
}
}
func TestComputeUsageSummary_MultiplePackages(t *testing.T) {
// 套餐1启用虚流量VirtualUsed=20, VirtualTotal=200
// 套餐2未启用虚流量VirtualUsed=10, VirtualTotal=50
// 汇总totalUsed=30, totalRemaining=(200+50)-30=220
usages := []*model.PackageUsage{
{
DataLimitMB: 100,
DataUsageMB: 40,
VirtualTotalMBSnapshot: 200,
DisplayGainRatioSnapshot: 0.5,
EnableVirtualDataSnapshot: true,
},
{
DataLimitMB: 50,
DataUsageMB: 10,
EnableVirtualDataSnapshot: false,
},
}
totalUsed, totalRemaining := computeUsageSummary(usages)
if totalUsed != 30.0 {
t.Errorf("期望 totalUsed=30.0,实际=%v", totalUsed)
}
if totalRemaining != 220.0 {
t.Errorf("期望 totalRemaining=220.0,实际=%v", totalRemaining)
}
}

View File

@@ -128,7 +128,7 @@ func (s *Service) CreateOrder(ctx context.Context, customerID uint, req *dto.Cli
}
skipCtx := context.WithValue(ctx, constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier))
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier), false)
if err != nil {
return nil, err
}
@@ -229,6 +229,18 @@ func (s *Service) CreateOrder(ctx context.Context, customerID uint, req *dto.Cli
}()
if forceRecharge.NeedForceRecharge {
// 强充第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// switch assetInfo.AssetType {
// case "card":
// if req.PaymentMethod != model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "单卡资产强充只支持支付宝支付")
// }
// case constants.ResourceTypeDevice:
// if req.PaymentMethod == model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "设备资产强充只支持微信支付")
// }
// }
if req.PaymentMethod == model.PaymentMethodAlipay {
// 支付宝强充:不需要 app_type / OpenID
activeConfig, err := s.wechatConfigService.GetActiveConfig(skipCtx)
@@ -1060,7 +1072,7 @@ func (s *Service) createAlipayForceRechargeOrder(
func (s *Service) ListOrders(ctx context.Context, customerID uint, req *dto.ClientOrderListRequest) ([]dto.ClientOrderListItem, int64, error) {
skipCtx := context.WithValue(ctx, constants.ContextKeySubordinateShopIDs, []uint{})
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier))
assetInfo, err := s.assetService.Resolve(skipCtx, strings.TrimSpace(req.Identifier), false)
if err != nil {
return nil, 0, err
}
@@ -1231,7 +1243,7 @@ func (s *Service) PayOrder(ctx context.Context, customerID uint, orderID uint, r
if err == nil && device != nil {
assetRealnamePolicy = device.RealnamePolicy
// 设备的 RealNameStatus 需要通过 Resolve 获取(从当前卡派生)
resolved, resolveErr := s.assetService.Resolve(skipCtx, device.VirtualNo)
resolved, resolveErr := s.assetService.Resolve(skipCtx, device.VirtualNo, false)
if resolveErr == nil && resolved != nil {
assetRealnameStatus = resolved.RealNameStatus
}
@@ -1241,6 +1253,20 @@ func (s *Service) PayOrder(ctx context.Context, customerID uint, orderID uint, r
return nil, errors.New(errors.CodeNeedRealname)
}
// 第三方支付方式与资产类型必须匹配:单卡只允许支付宝,设备只允许微信(已暂时注释)
// if req.PaymentMethod == model.PaymentMethodWechat || req.PaymentMethod == model.PaymentMethodAlipay {
// switch order.OrderType {
// case model.OrderTypeSingleCard:
// if req.PaymentMethod != model.PaymentMethodAlipay {
// return nil, errors.New(errors.CodeInvalidParam, "单卡资产只支持支付宝支付")
// }
// case model.OrderTypeDevice:
// if req.PaymentMethod != model.PaymentMethodWechat {
// return nil, errors.New(errors.CodeInvalidParam, "设备资产只支持微信支付")
// }
// }
// }
switch req.PaymentMethod {
case model.PaymentMethodWallet:
if err := s.orderPaymentService.WalletPay(skipCtx, orderID, model.BuyerTypePersonal, customerID); err != nil {

View File

@@ -67,6 +67,16 @@ func (s *DeviceImportTaskStore) UpdateStatus(ctx context.Context, id uint, statu
return s.db.WithContext(ctx).Model(&model.DeviceImportTask{}).Where("id = ?", id).Updates(updates).Error
}
// UpdateProgress 实时更新导入进度计数(不更新详情列表,用于批量处理中间状态)
func (s *DeviceImportTaskStore) UpdateProgress(ctx context.Context, id uint, successCount, skipCount, failCount int) error {
return s.db.WithContext(ctx).Model(&model.DeviceImportTask{}).Where("id = ?", id).Updates(map[string]any{
"success_count": successCount,
"skip_count": skipCount,
"fail_count": failCount,
"updated_at": time.Now(),
}).Error
}
func (s *DeviceImportTaskStore) UpdateResult(ctx context.Context, id uint, totalCount, successCount, skipCount, failCount, warningCount int, skippedItems, failedItems, warningItems model.ImportResultItems) error {
updates := map[string]any{
"total_count": totalCount,

View File

@@ -72,6 +72,16 @@ func (s *IotCardImportTaskStore) UpdateStatus(ctx context.Context, id uint, stat
return s.db.WithContext(ctx).Model(&model.IotCardImportTask{}).Where("id = ?", id).Updates(updates).Error
}
// UpdateProgress 实时更新导入进度计数(不更新详情列表,用于批量处理中间状态)
func (s *IotCardImportTaskStore) UpdateProgress(ctx context.Context, id uint, successCount, skipCount, failCount int) error {
return s.db.WithContext(ctx).Model(&model.IotCardImportTask{}).Where("id = ?", id).Updates(map[string]interface{}{
"success_count": successCount,
"skip_count": skipCount,
"fail_count": failCount,
"updated_at": time.Now(),
}).Error
}
func (s *IotCardImportTaskStore) UpdateResult(ctx context.Context, id uint, successCount, skipCount, failCount int, skippedItems, failedItems model.ImportResultItems) error {
updates := map[string]interface{}{
"success_count": successCount,

View File

@@ -301,6 +301,40 @@ func (s *PackageUsageStore) ListByCarrier(ctx context.Context, carrierType strin
return usages, nil
}
// GetCurrentGenerationUsagesForSummary 查询当前世代内所有已激活过的套餐记录(含加油包),用于流量汇总。
// 当前世代定义status∈{1,2,3,4} 中 generation 最大值所在的那一批记录。
// 返回计算所需字段DataUsageMB、DataLimitMB、VirtualTotalMBSnapshot、DisplayGainRatioSnapshot、EnableVirtualDataSnapshot。
func (s *PackageUsageStore) GetCurrentGenerationUsagesForSummary(ctx context.Context, carrierType string, carrierID uint) ([]*model.PackageUsage, error) {
var usages []*model.PackageUsage
// 子查询取当前世代编号
subQuery := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Select("MAX(generation)").
Where("status IN ?", []int{
constants.PackageUsageStatusActive,
constants.PackageUsageStatusDepleted,
constants.PackageUsageStatusExpired,
constants.PackageUsageStatusInvalidated,
})
subQuery = applyPackageUsageCarrierFilter(subQuery, carrierType, carrierID)
query := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Select("data_usage_mb, data_limit_mb, virtual_total_mb_snapshot, display_gain_ratio_snapshot, enable_virtual_data_snapshot").
Where("status IN ?", []int{
constants.PackageUsageStatusActive,
constants.PackageUsageStatusDepleted,
constants.PackageUsageStatusExpired,
constants.PackageUsageStatusInvalidated,
}).
Where("generation = (?)", subQuery)
query = applyPackageUsageCarrierFilter(query, carrierType, carrierID)
if err := query.Find(&usages).Error; err != nil {
return nil, err
}
return usages, nil
}
func applyPackageUsageCarrierFilter(query *gorm.DB, carrierType string, carrierID uint) *gorm.DB {
if carrierType == "iot_card" {
return query.Where("iot_card_id = ?", carrierID)

View File

@@ -85,7 +85,18 @@ func (h *DeviceImportHandler) HandleDeviceImport(ctx context.Context, task *asyn
return asynq.SkipRetry
}
if importTask.Status != model.ImportTaskStatusPending {
switch importTask.Status {
case model.ImportTaskStatusPending:
// 正常首次处理
case model.ImportTaskStatusProcessing:
// 上次 worker 中途中断(重启/崩溃),重置计数重新处理
// 已入库的设备会被 ExistsByDeviceNoBatch 识别为 skip不会重复写入
h.logger.Warn("检测到导入任务上次处理中途中断,重置进度重新处理",
zap.Uint("task_id", payload.TaskID),
)
h.importTaskStore.UpdateProgress(ctx, importTask.ID, 0, 0, 0)
default:
// 已完成或失败,不重复处理
h.logger.Info("导入任务已处理,跳过",
zap.Uint("task_id", payload.TaskID),
zap.Int("status", importTask.Status),
@@ -185,6 +196,8 @@ func (h *DeviceImportHandler) processImport(ctx context.Context, task *model.Dev
end := min(i+deviceBatchSize, len(rows))
batch := rows[i:end]
h.processBatch(ctx, task, batch, result)
// 每批完成后实时更新进度计数,让前端可以看到处理进度
h.importTaskStore.UpdateProgress(ctx, task.ID, result.successCount, result.skipCount, result.failCount)
}
return result

View File

@@ -41,15 +41,14 @@ type PollingCallback interface {
}
type IotCardImportHandler struct {
db *gorm.DB
redis *redis.Client
importTaskStore *postgres.IotCardImportTaskStore
iotCardStore *postgres.IotCardStore
assetWalletStore *postgres.AssetWalletStore
assetIdentifierStore *postgres.AssetIdentifierStore
storageService *storage.Service
pollingCallback PollingCallback
logger *zap.Logger
db *gorm.DB
redis *redis.Client
importTaskStore *postgres.IotCardImportTaskStore
iotCardStore *postgres.IotCardStore
assetWalletStore *postgres.AssetWalletStore
storageService *storage.Service
pollingCallback PollingCallback
logger *zap.Logger
}
func NewIotCardImportHandler(
@@ -58,21 +57,19 @@ func NewIotCardImportHandler(
importTaskStore *postgres.IotCardImportTaskStore,
iotCardStore *postgres.IotCardStore,
assetWalletStore *postgres.AssetWalletStore,
assetIdentifierStore *postgres.AssetIdentifierStore,
storageSvc *storage.Service,
pollingCallback PollingCallback,
logger *zap.Logger,
) *IotCardImportHandler {
return &IotCardImportHandler{
db: db,
redis: redis,
importTaskStore: importTaskStore,
iotCardStore: iotCardStore,
assetWalletStore: assetWalletStore,
assetIdentifierStore: assetIdentifierStore,
storageService: storageSvc,
pollingCallback: pollingCallback,
logger: logger,
db: db,
redis: redis,
importTaskStore: importTaskStore,
iotCardStore: iotCardStore,
assetWalletStore: assetWalletStore,
storageService: storageSvc,
pollingCallback: pollingCallback,
logger: logger,
}
}
@@ -95,7 +92,18 @@ func (h *IotCardImportHandler) HandleIotCardImport(ctx context.Context, task *as
return asynq.SkipRetry
}
if importTask.Status != model.ImportTaskStatusPending {
switch importTask.Status {
case model.ImportTaskStatusPending:
// 正常首次处理
case model.ImportTaskStatusProcessing:
// 上次 worker 中途中断(重启/崩溃),重置计数重新处理
// 已入库的卡会被 ExistsByICCIDBatch 识别为 skip不会重复写入
h.logger.Warn("检测到导入任务上次处理中途中断,重置进度重新处理",
zap.Uint("task_id", payload.TaskID),
)
h.importTaskStore.UpdateProgress(ctx, importTask.ID, 0, 0, 0)
default:
// 已完成或失败,不重复处理
h.logger.Info("导入任务已处理,跳过",
zap.Uint("task_id", payload.TaskID),
zap.Int("status", importTask.Status),
@@ -218,6 +226,8 @@ func (h *IotCardImportHandler) processImport(ctx context.Context, task *model.Io
end := min(i+batchSize, len(cards))
batch := cards[i:end]
h.processBatch(ctx, task, batch, i+1, result)
// 每批完成后实时更新进度计数,让前端可以看到处理进度
h.importTaskStore.UpdateProgress(ctx, task.ID, result.successCount, result.skipCount, result.failCount)
}
return result
@@ -341,7 +351,14 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
}
now := time.Now()
createdCards := make([]*model.IotCard, 0, len(finalCards))
// 构建待批量写入的卡列表,同时记录原始信息用于失败回溯
type pendingEntry struct {
item model.CardItem
meta cardMeta
}
pending := make([]pendingEntry, 0, len(finalCards))
iotCards := make([]*model.IotCard, 0, len(finalCards))
for _, card := range finalCards {
meta := cardMetaMap[card.ICCID]
@@ -386,43 +403,63 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
iotCard.CreatedAt = now
iotCard.UpdatedAt = now
txErr := h.db.Transaction(func(tx *gorm.DB) error {
if err := tx.Create(iotCard).Error; err != nil {
return err
}
txIdentifierStore := postgres.NewAssetIdentifierStore(tx)
if err := txIdentifierStore.Register(ctx, card.ICCID, model.AssetTypeIotCard, iotCard.ID); err != nil {
return fmt.Errorf("ICCID 已被占用: %s", card.ICCID)
}
if card.VirtualNo != "" {
if err := txIdentifierStore.Register(ctx, card.VirtualNo, model.AssetTypeIotCard, iotCard.ID); err != nil {
return fmt.Errorf("虚拟号已被占用: %s", card.VirtualNo)
}
}
return nil
})
if txErr != nil {
h.logger.Error("创建 IoT 卡失败",
zap.String("iccid", card.ICCID),
zap.Error(txErr),
)
result.failedItems = append(result.failedItems, model.ImportResultItem{
Line: meta.line,
ICCID: card.ICCID,
MSISDN: meta.msisdn,
Reason: txErr.Error(),
})
result.failCount++
continue
}
createdCards = append(createdCards, iotCard)
result.successCount++
iotCards = append(iotCards, iotCard)
pending = append(pending, pendingEntry{item: card, meta: meta})
}
h.batchCreateWallets(ctx, createdCards)
if len(iotCards) == 0 {
return
}
// 整批一个事务:批量写入 iot_card + asset_identifier
// 相比逐卡事务,将 N 个事务压缩为 1 个,大幅减少数据库往返
txErr := h.db.Transaction(func(tx *gorm.DB) error {
// 批量插入 iot_cardGORM 会通过 RETURNING 回填所有 ID
if err := tx.CreateInBatches(&iotCards, 500).Error; err != nil {
return err
}
// 用回填的 ID 构建 asset_identifier 列表
identifiers := make([]*model.AssetIdentifier, 0, len(iotCards)*2)
for _, c := range iotCards {
identifiers = append(identifiers, &model.AssetIdentifier{
Identifier: c.ICCID,
AssetType: model.AssetTypeIotCard,
AssetID: c.ID,
})
if c.VirtualNo != "" {
identifiers = append(identifiers, &model.AssetIdentifier{
Identifier: c.VirtualNo,
AssetType: model.AssetTypeIotCard,
AssetID: c.ID,
})
}
}
return tx.CreateInBatches(&identifiers, 500).Error
})
if txErr != nil {
h.logger.Error("批量创建 IoT 卡失败",
zap.Error(txErr),
zap.Int("batch_size", len(iotCards)),
)
for _, p := range pending {
result.failedItems = append(result.failedItems, model.ImportResultItem{
Line: p.meta.line,
ICCID: p.item.ICCID,
MSISDN: p.meta.msisdn,
Reason: "批量写入失败: " + txErr.Error(),
})
result.failCount++
}
return
}
result.successCount += len(iotCards)
h.batchCreateWallets(ctx, iotCards)
if h.pollingCallback != nil {
h.pollingCallback.OnBatchCardsCreated(ctx, createdCards)
h.pollingCallback.OnBatchCardsCreated(ctx, iotCards)
}
}

View File

@@ -88,7 +88,6 @@ func (h *Handler) registerIotCardImportHandler() {
h.workerResult.Stores.IotCardImportTask,
h.workerResult.Stores.IotCard,
h.workerResult.Stores.AssetWallet,
h.workerResult.Stores.AssetIdentifier,
h.storage,
h.pollingCallback,
h.logger,

View File

@@ -6,12 +6,29 @@
from __future__ import annotations
import contextlib
import time
from dataclasses import dataclass
from decimal import Decimal
from typing import Any, Iterable, Iterator, Optional
BATCH_SIZE = 200
# 重构后分阶段索引查询已无大 JOIN,1000 为安全默认值
BATCH_SIZE = 1000
class _StageTimer:
"""阶段耗时计时器,记录迁移各阶段用时。"""
def __init__(self, func_name: str) -> None:
self._func = func_name
self._t = time.perf_counter()
def log(self, stage: str, **kw: Any) -> None:
elapsed = time.perf_counter() - self._t
parts = " ".join(f"{k}={v}" for k, v in kw.items())
suffix = f" {parts}" if parts else ""
print(f"[legacy] {self._func}.{stage}:{suffix} elapsed={elapsed:.1f}s")
self._t = time.perf_counter()
def _normalize_iccid_pairs(iccid_pairs: Iterable[tuple]) -> list[tuple[str, str, bool]]:
@@ -236,14 +253,35 @@ def _to_mb_decimal(value) -> Decimal:
return Decimal("0")
def _build_card_lookup(
pairs: list[tuple[str, str, bool]],
) -> dict[str, tuple[str, int]]:
"""构建 tbl_card 查询键 → (完整 ICCID, rank) 映射。
rank=0 表示完整精确命中,rank=1 表示 19 位兜底命中。
"""
by_full: dict[str, tuple[str, int]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
by_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
by_full[i19] = (card_key, 1)
return by_full
# ---------------- 查询函数 ----------------
def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPackage]:
def fetch_current_packages(
conn,
iccids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyPackage]:
"""查每张卡的当前生效主套餐:status=1、type!=1 且 expire_date 最大。
返回: iccid → LegacyPackage(找不到的 iccid 不在结果中)
"""
timer = _StageTimer("fetch_current_packages")
iccid_list = sorted({i for i in iccids if i})
out: dict[str, LegacyPackage] = {}
if not iccid_list:
@@ -266,11 +304,15 @@ def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPacka
ORDER BY iccid_mark ASC, expire_date DESC
"""
seen: set[str] = set()
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
rows += 1
if not iccid or iccid in seen:
continue
seen.add(iccid)
@@ -284,15 +326,21 @@ def fetch_current_packages(conn, iccids: Iterable[str]) -> dict[str, LegacyPacka
flow_size_mb=int(row.get("flow_size") or 0),
status=int(row.get("status") or 0),
)
timer.log("tbl_card_life", keys=len(iccid_list), batches=batches, rows=rows, found=len(out))
return out
def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, list[LegacyPackageLifecycle]]:
def fetch_package_lifecycles(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> dict[str, list[LegacyPackageLifecycle]]:
"""查当前生效和未生效待生效正式套餐生命周期。
只返回可迁移范围(active/pending)和需要在审核文件中说明的跳过记录。
加油包(type=1)和已过期记录不生成套餐 SQL,但保留 skip_reason 供审核产物输出。
"""
timer = _StageTimer("fetch_package_lifecycles")
pairs = _normalize_iccid_pairs(iccid_pairs)
out: dict[str, list[LegacyPackageLifecycle]] = {}
if not pairs:
@@ -328,8 +376,11 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
COALESCE(expire_date, '9999-12-31') ASC,
id ASC
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row.get("iccid_mark") or ""
@@ -338,6 +389,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
iccid = query_to_full.get(legacy_iccid)
if not iccid:
continue
rows += 1
start = row.get("start_date")
expire = row.get("expire_date")
start_str = start.strftime("%Y-%m-%d %H:%M:%S") if start else None
@@ -363,6 +415,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
skip_reason=skip_reason,
)
out.setdefault(iccid, []).append(item)
timer.log("tbl_card_life", keys=len(iccid_list), batches=batches, rows=rows)
next_sql = """
SELECT
@@ -384,8 +437,11 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
COALESCE(effect_complete_time, '9999-12-31') ASC,
id ASC
"""
next_rows = 0
next_batches = 0
with conn.cursor() as cur:
for batch in _chunks(iccid_list):
for batch in _chunks(iccid_list, batch_size):
next_batches += 1
cur.execute(next_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row.get("iccid_mark") or ""
@@ -394,6 +450,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
iccid = query_to_full.get(legacy_iccid)
if not iccid:
continue
next_rows += 1
start = row.get("start_date")
expire = row.get("expire_date")
start_str = start.strftime("%Y-%m-%d %H:%M:%S") if start else None
@@ -417,6 +474,7 @@ def fetch_package_lifecycles(conn, iccid_pairs: Iterable[tuple]) -> dict[str, li
migration_status=migration_status,
skip_reason=skip_reason,
))
timer.log("tbl_next_month_card_life", keys=len(iccid_list), batches=next_batches, rows=next_rows)
return out
@@ -448,13 +506,18 @@ def _classify_next_month_lifecycle(status: int) -> tuple[str, str]:
return "skipped", f"奇成次月待生效状态 {status} 不在迁移范围"
def fetch_card_usage(conn, iccid_pairs: Iterable[tuple]) -> dict[str, LegacyCardUsage]:
"""查每张卡的累计已用流量,并同步取当前生效正式套餐的 flow_add_discount。
def fetch_card_usage(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyCardUsage]:
"""查每张卡的累计已用流量,分阶段索引查询替代大 JOIN。
奇成 ``tbl_card.total_bytes_cnt`` 是虚用量(已被套餐虚比加成)。
这里 JOIN 当前生效正式套餐(``tbl_card_life`` status=1,type!=1 且 ``expire_date`` 最大)
+ ``tbl_set_meal`` 取 ``flow_add_discount``,放进 ``LegacyCardUsage``;
上层通过 ``real_used_mb_floor`` 反算真用量写入新系统。
阶段:
1. tbl_card — iccid_mark + total_bytes_cnt
2. tbl_card_life — 当前生效正式套餐 meal_id(Python 选 expire_date 最大)
3. tbl_set_meal — flow_add_discount(按 meal_id 精确查)
4. Python 组装 LegacyCardUsage
Args:
iccid_pairs: (iccid_19, iccid_20, allow_iccid_19_lookup) 列表。
@@ -462,88 +525,119 @@ def fetch_card_usage(conn, iccid_pairs: Iterable[tuple]) -> dict[str, LegacyCard
Returns:
{完整 ICCID: LegacyCardUsage}。
"""
timer = _StageTimer("fetch_card_usage")
pairs = _normalize_iccid_pairs(iccid_pairs)
out: dict[str, LegacyCardUsage] = {}
if not pairs:
return out
query_to_full: dict[str, tuple[str, int]] = {}
related_keys_by_full: dict[str, set[str]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
# tbl_card 主表同时查优先键和兜底键,结果选择时按 rank:
# 0=业务方完整 ICCID 精确命中,1=20 位卡在奇成主表只存 19 位时兜底。
query_to_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
query_to_full[i19] = (card_key, 1)
if not i20 or allow_i19_lookup:
related_keys_by_full.setdefault(card_key, set()).add(i19)
else:
related_keys_by_full.setdefault(card_key, set())
if i20:
related_keys_by_full[card_key].add(i20)
full_set = sorted(query_to_full.keys())
by_full = _build_card_lookup(pairs)
full_set = sorted(by_full.keys())
# tbl_card 主表用 iccid_mark IN 全长精确匹配;从表只在主表本身是 19 位时
# 按前缀兜底,避免同前缀 20 位卡互相串套餐。
sql = """
# ── 阶段1: tbl_card ───────────────────────────────────────────────────────
card_sql = """
SELECT
c.iccid_mark,
c.total_bytes_cnt,
COALESCE(sm.flow_add_discount, 0) AS flow_add_discount,
CASE WHEN cl.iccid_mark IS NOT NULL THEN 1 ELSE 0 END AS has_active_package
FROM tbl_card c
LEFT JOIN (
SELECT cl1.iccid_mark, cl1.meal_id
FROM tbl_card_life cl1
INNER JOIN (
SELECT iccid_mark, MAX(expire_date) AS max_expire
FROM tbl_card_life
WHERE status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
AND iccid_mark IN %s
GROUP BY iccid_mark
) latest
ON latest.iccid_mark = cl1.iccid_mark
AND latest.max_expire = cl1.expire_date
WHERE cl1.status = 1
AND COALESCE(CAST(cl1.type AS CHAR), '') <> '1'
) cl ON (
cl.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(cl.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN tbl_set_meal sm ON sm.id = cl.meal_id
WHERE c.iccid_mark IN %s
iccid_mark,
COALESCE(total_bytes_cnt, 0) AS total_bytes_cnt
FROM tbl_card
WHERE iccid_mark IN %s
"""
hits: dict[str, list[tuple[int, dict]]] = {}
card_rows = 0
card_batches = 0
with conn.cursor() as cur:
for batch in _chunks(full_set):
batch_keys = {query_to_full[x][0] for x in batch}
related_batch = sorted({x for key in batch_keys for x in related_keys_by_full[key]})
cur.execute(sql, (tuple(related_batch), tuple(batch)))
for batch in _chunks(full_set, batch_size):
card_batches += 1
cur.execute(card_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row["iccid_mark"]
if not legacy_iccid:
continue
matched = query_to_full.get(legacy_iccid)
matched = by_full.get(legacy_iccid)
if matched is None:
continue
card_key, rank = matched
hits.setdefault(card_key, []).append((rank, row))
card_rows += 1
timer.log("tbl_card", keys=len(full_set), batches=card_batches, rows=card_rows)
# 取最优命中行,构建 legacy_iccid 列表
card_data: dict[str, dict] = {} # card_key → best tbl_card row
for card_key, rows in hits.items():
best_rank = min(rank for rank, _row in rows)
row = next(row for rank, row in rows if rank == best_rank)
best_rank = min(r for r, _ in rows)
card_data[card_key] = next(row for r, row in rows if r == best_rank)
if not card_data:
return out
legacy_iccids = sorted({row["iccid_mark"] for row in card_data.values()})
# ── 阶段2: tbl_card_life 当前生效正式套餐(精确匹配,Python 选最新) ─────────
life_sql = """
SELECT
iccid_mark,
COALESCE(meal_id, '') AS meal_id
FROM tbl_card_life
WHERE iccid_mark IN %s
AND status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
ORDER BY iccid_mark ASC, expire_date DESC, id DESC
"""
current_meal_by_iccid: dict[str, str] = {} # legacy_iccid → meal_id
life_rows = 0
life_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
life_batches += 1
cur.execute(life_sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
life_rows += 1
if iccid not in current_meal_by_iccid:
current_meal_by_iccid[iccid] = row.get("meal_id") or ""
timer.log("tbl_card_life", keys=len(legacy_iccids), batches=life_batches, rows=life_rows)
# ── 阶段3: tbl_set_meal flow_add_discount(按 distinct meal_id 查) ─────────
meal_ids = sorted({mid for mid in current_meal_by_iccid.values() if mid})
discount_by_meal: dict[str, Decimal] = {}
if meal_ids:
meal_sql = "SELECT id, COALESCE(flow_add_discount, 0) AS flow_add_discount FROM tbl_set_meal WHERE id IN %s"
meal_batches = 0
with conn.cursor() as cur:
for batch in _chunks(meal_ids, batch_size):
meal_batches += 1
cur.execute(meal_sql, (tuple(batch),))
for row in cur.fetchall():
mid = str(row["id"] or "")
if mid:
discount_by_meal[mid] = _to_mb_decimal(row.get("flow_add_discount"))
timer.log("tbl_set_meal", meal_ids=len(meal_ids), batches=meal_batches)
else:
timer.log("tbl_set_meal", meal_ids=0, skipped=True)
# ── 阶段4: Python 组装 ────────────────────────────────────────────────────
for card_key, base_row in card_data.items():
legacy_iccid = base_row["iccid_mark"]
meal_id = current_meal_by_iccid.get(legacy_iccid, "")
has_active = bool(meal_id or legacy_iccid in current_meal_by_iccid)
discount = discount_by_meal.get(str(meal_id), Decimal("0")) if meal_id else Decimal("0")
out[card_key] = LegacyCardUsage(
iccid=card_key,
virtual_used_mb=_to_mb_decimal(row.get("total_bytes_cnt")),
flow_add_discount_pct=_to_mb_decimal(row.get("flow_add_discount")),
has_active_package=bool(row.get("has_active_package")),
virtual_used_mb=_to_mb_decimal(base_row.get("total_bytes_cnt")),
flow_add_discount_pct=discount,
has_active_package=has_active,
)
timer.log("assemble", cards=len(out))
return out
def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, LegacyCommissionAccount]:
def fetch_commission_accounts(
conn,
agent_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyCommissionAccount]:
"""查代理佣金账户。"""
timer = _StageTimer("fetch_commission_accounts")
agent_list = sorted({a for a in agent_ids if a})
out: dict[str, LegacyCommissionAccount] = {}
if not agent_list:
@@ -557,11 +651,15 @@ def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, Legac
FROM tbl_agent_commission_account
WHERE agent_id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(agent_list):
for batch in _chunks(agent_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
aid = row["agent_id"]
rows += 1
if not aid:
continue
out[aid] = LegacyCommissionAccount(
@@ -572,107 +670,69 @@ def fetch_commission_accounts(conn, agent_ids: Iterable[str]) -> dict[str, Legac
total_commission_amount_fen=_to_fen(row.get("total_commision_amount")),
total_draw_amount_fen=_to_fen(row.get("total_draw_amount")),
)
timer.log("tbl_agent_commission_account", agents=len(agent_list), batches=batches, rows=rows)
return out
def fetch_card_meta(
conn,
iccid_pairs: Iterable[tuple],
batch_size: int = BATCH_SIZE,
) -> tuple[dict[str, LegacyCardMeta], set[str]]:
"""查每张卡的元数据(卡本体 + 当前生效套餐 + 虚拟号)
"""查每张卡的元数据,分阶段索引查询替代大 JOIN
阶段:
1. tbl_card — 卡基础字段 + 去重检测
2. tbl_vendor_category — 运营商分类名称(按 distinct category 查)
3. tbl_card_life — 当前生效正式套餐(Python 选 expire_date 最大,精确匹配)
4. tbl_set_meal + tbl_set_meal_series — 套餐系列(按 distinct meal_id 查)
5. tbl_virtual_number — 虚拟号(精确匹配,不做 LEFT() 前缀扫描)
6. Python 组装 LegacyCardMeta
匹配策略:
- tbl_card 主表同时查优先键和显式允许的兜底键:有 iccid_20 时优先
20 位精确命中;只有业务方也显式填了 iccid_19 时,才允许退回 19 位。
只填 iccid_20 的固定 20 位卡绝不拿 19 位前缀查主表。
- tbl_card_life / tbl_virtual_number 从表在主表本身是 19 位时才用
LEFT(iccid_mark, 19) 兜底,避免同前缀 20 位卡互相串数据
- tbl_card 主表同时查优先键和显式允许的兜底键;20 位精确命中优先
- 从表(card_life / virtual_number)用 tbl_card 实际返回的 iccid_mark 精确查询
- 不再使用 LEFT(vn.iccid_mark, 19) 前缀匹配,避免全表扫描
Args:
iccid_pairs: (iccid_19, iccid_20, allow_iccid_19_lookup) 列表。
Returns:
(metas, duplicate_iccids):
- metas: {完整 ICCID: LegacyCardMeta} 单匹结果
- duplicate_iccids: 一份业务方完整 ICCID 命中奇成 tbl_card 多条记录的 set,
这些卡不进 metas,sql_builder 会写 errors.csv 阻断
- metas: {完整 ICCID: LegacyCardMeta}
- duplicate_iccids: 一份业务方 ICCID 命中多条 tbl_card 记录的 set
"""
timer = _StageTimer("fetch_card_meta")
pairs = _normalize_iccid_pairs(iccid_pairs)
if not pairs:
return {}, set()
by_full: dict[str, tuple[str, int]] = {} # tbl_card 查询键 → (完整 ICCID, rank)
related_keys_by_full: dict[str, set[str]] = {}
for i19, i20, allow_i19_lookup in pairs:
card_key = i20 or i19
by_full[card_key] = (card_key, 0)
if i20 and allow_i19_lookup:
by_full[i19] = (card_key, 1)
if not i20 or allow_i19_lookup:
related_keys_by_full.setdefault(card_key, set()).add(i19)
else:
related_keys_by_full.setdefault(card_key, set())
if i20:
related_keys_by_full[card_key].add(i20)
by_full = _build_card_lookup(pairs)
full_set = sorted(by_full.keys())
sql = """
# ── 阶段1: tbl_card ───────────────────────────────────────────────────────
card_sql = """
SELECT
c.id AS card_id,
c.iccid_mark AS iccid,
COALESCE(c.account_id, '') AS account_id,
COALESCE(c.account_name, '') AS account_name,
COALESCE(c.category, 0) AS category,
COALESCE(vc.category_name, '') AS category_name,
COALESCE(c.agent_id, '') AS agent_id,
COALESCE(c.agent_name, '') AS agent_name,
COALESCE(c.phone, '') AS phone,
COALESCE(vn.vcode, '') AS vcode,
COALESCE(cl.meal_id, '') AS meal_id,
COALESCE(cl.meal_name, '') AS meal_name,
COALESCE(CAST(cl.meal_type AS CHAR), '') AS meal_type,
cl.expire_date AS expire_date,
COALESCE(sm.meal_series_id, '') AS series_id,
COALESCE(sms.name, '') AS series_name
FROM tbl_card c
LEFT JOIN tbl_vendor_category vc ON vc.category = c.category
LEFT JOIN tbl_virtual_number vn ON (
vn.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(vn.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN (
SELECT cl1.iccid_mark, cl1.meal_id, cl1.meal_name, cl1.type AS meal_type, cl1.expire_date
FROM tbl_card_life cl1
INNER JOIN (
SELECT iccid_mark, MAX(expire_date) AS max_expire
FROM tbl_card_life
WHERE status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
AND iccid_mark IN %s
GROUP BY iccid_mark
) latest
ON latest.iccid_mark = cl1.iccid_mark
AND latest.max_expire = cl1.expire_date
WHERE cl1.status = 1
AND COALESCE(CAST(cl1.type AS CHAR), '') <> '1'
) cl ON (
cl.iccid_mark = c.iccid_mark
OR (CHAR_LENGTH(c.iccid_mark) = 19 AND LEFT(cl.iccid_mark, 19) = c.iccid_mark)
)
LEFT JOIN tbl_set_meal sm ON sm.id = cl.meal_id
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE c.iccid_mark IN %s
id AS card_id,
iccid_mark,
COALESCE(account_id, '') AS account_id,
COALESCE(account_name, '') AS account_name,
COALESCE(category, 0) AS category,
COALESCE(agent_id, '') AS agent_id,
COALESCE(agent_name, '') AS agent_name,
COALESCE(phone, '') AS phone
FROM tbl_card
WHERE iccid_mark IN %s
"""
# 收集每个完整 ICCID 命中的奇成行(可能多条 = 脏数据)
hits: dict[str, list[tuple[int, dict]]] = {}
card_rows = 0
card_batches = 0
with conn.cursor() as cur:
for batch in _chunks(full_set):
batch_keys = {by_full[x][0] for x in batch}
related_batch = sorted({x for key in batch_keys for x in related_keys_by_full[key]})
cur.execute(sql, (tuple(related_batch), tuple(batch)))
for batch in _chunks(full_set, batch_size):
card_batches += 1
cur.execute(card_sql, (tuple(batch),))
for row in cur.fetchall():
legacy_iccid = row["iccid"]
legacy_iccid = row["iccid_mark"]
if not legacy_iccid:
continue
matched = by_full.get(legacy_iccid)
@@ -680,45 +740,162 @@ def fetch_card_meta(
continue
card_key, rank = matched
hits.setdefault(card_key, []).append((rank, row))
card_rows += 1
timer.log("tbl_card", keys=len(full_set), batches=card_batches, rows=card_rows)
metas: dict[str, LegacyCardMeta] = {}
# 去重:同一完整 ICCID 命中多条不同 card_id → duplicates
metas_base: dict[str, dict] = {} # card_key → best tbl_card row
duplicates: set[str] = set()
categories: set[int] = set()
for card_key, rows in hits.items():
best_rank = min(rank for rank, _row in rows)
best_rows = [row for rank, row in rows if rank == best_rank]
# 同一张 tbl_card 可能因 card_life / virtual_number 前缀兜底 JOIN 出多行,
# 这不是主表重复;只有不同 card_id 才算同一业务 ICCID 命中多张卡。
best_rank = min(r for r, _ in rows)
best_rows = [row for r, row in rows if r == best_rank]
rows_by_card_id: dict[str, dict] = {}
for row in best_rows:
rows_by_card_id[str(row.get("card_id") or row.get("iccid") or "")] = row
rows_by_card_id[str(row.get("card_id") or row.get("iccid_mark") or "")] = row
if len(rows_by_card_id) > 1:
duplicates.add(card_key)
continue
row = next(iter(rows_by_card_id.values()))
expire = row.get("expire_date")
metas_base[card_key] = row
categories.add(int(row.get("category") or 0))
if not metas_base:
return {}, duplicates
# legacy_iccids: tbl_card 实际存储的 iccid_mark 值(用于从表精确查询)
legacy_iccids = sorted({row["iccid_mark"] for row in metas_base.values()})
# ── 阶段2: tbl_vendor_category(distinct category,通常极少几十个) ─────────
category_names: dict[int, str] = {}
cat_batches = 0
if categories:
cat_sql = """
SELECT category, COALESCE(category_name, '') AS category_name
FROM tbl_vendor_category
WHERE category IN %s
"""
cat_list = sorted(categories)
with conn.cursor() as cur:
for batch in _chunks(cat_list, batch_size):
cat_batches += 1
cur.execute(cat_sql, (tuple(batch),))
for row in cur.fetchall():
category_names[int(row["category"])] = row.get("category_name") or ""
timer.log("tbl_vendor_category", categories=len(categories), batches=cat_batches)
# ── 阶段3: tbl_card_life 当前生效正式套餐(精确匹配,Python 选最新) ─────────
# 精确用 legacy_iccid 查询;不再做 LEFT(iccid_mark, 19) 前缀扫描
life_sql = """
SELECT
iccid_mark,
COALESCE(meal_id, '') AS meal_id,
COALESCE(meal_name, '') AS meal_name,
COALESCE(CAST(type AS CHAR), '') AS meal_type,
expire_date
FROM tbl_card_life
WHERE iccid_mark IN %s
AND status = 1
AND COALESCE(CAST(type AS CHAR), '') <> '1'
ORDER BY iccid_mark ASC, expire_date DESC, id DESC
"""
current_meals: dict[str, dict] = {} # legacy_iccid → 当前生效套餐行
meal_ids: set[str] = set()
life_rows = 0
life_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
life_batches += 1
cur.execute(life_sql, (tuple(batch),))
for row in cur.fetchall():
iccid = row["iccid_mark"]
life_rows += 1
if iccid not in current_meals:
current_meals[iccid] = row
if row.get("meal_id"):
meal_ids.add(row["meal_id"])
timer.log("tbl_card_life", keys=len(legacy_iccids), batches=life_batches, rows=life_rows, found=len(current_meals))
# ── 阶段4: tbl_set_meal + tbl_set_meal_series(按 distinct meal_id) ───────
meal_series: dict[str, tuple[str, str]] = {} # meal_id → (series_id, series_name)
meal_batches = 0
if meal_ids:
meal_sql = """
SELECT
sm.id AS meal_id,
COALESCE(sm.meal_series_id, '') AS series_id,
COALESCE(sms.name, '') AS series_name
FROM tbl_set_meal sm
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE sm.id IN %s
"""
meal_id_list = sorted(meal_ids)
with conn.cursor() as cur:
for batch in _chunks(meal_id_list, batch_size):
meal_batches += 1
cur.execute(meal_sql, (tuple(batch),))
for row in cur.fetchall():
mid = str(row["meal_id"] or "")
if mid:
meal_series[mid] = (row.get("series_id") or "", row.get("series_name") or "")
timer.log("tbl_set_meal", meal_ids=len(meal_ids), batches=meal_batches)
# ── 阶段5: tbl_virtual_number(精确匹配,不做前缀扫描) ─────────────────────
virtual_nos: dict[str, str] = {} # legacy_iccid → vcode
vn_rows = 0
vn_batches = 0
with conn.cursor() as cur:
for batch in _chunks(legacy_iccids, batch_size):
vn_batches += 1
cur.execute(
"SELECT iccid_mark, COALESCE(vcode, '') AS vcode FROM tbl_virtual_number WHERE iccid_mark IN %s",
(tuple(batch),),
)
for row in cur.fetchall():
iccid = row["iccid_mark"]
vn_rows += 1
if iccid:
virtual_nos[iccid] = row.get("vcode") or ""
timer.log("tbl_virtual_number", keys=len(legacy_iccids), batches=vn_batches, rows=vn_rows)
# ── 阶段6: Python 组装 ────────────────────────────────────────────────────
metas: dict[str, LegacyCardMeta] = {}
for card_key, base_row in metas_base.items():
legacy_iccid = base_row["iccid_mark"]
meal_row = current_meals.get(legacy_iccid, {})
expire = meal_row.get("expire_date")
expire_str = expire.strftime("%Y-%m-%d %H:%M:%S") if expire else None
meal_id = meal_row.get("meal_id") or ""
series_id, series_name = meal_series.get(str(meal_id), ("", ""))
metas[card_key] = LegacyCardMeta(
legacy_raw_iccid=row["iccid"],
account_id=row.get("account_id") or "",
account_name=row.get("account_name") or "",
category=int(row.get("category") or 0),
category_name=row.get("category_name") or "",
agent_id=row.get("agent_id") or "",
agent_name=row.get("agent_name") or "",
msisdn=row.get("phone") or "",
virtual_no=row.get("vcode") or "",
current_meal_id=row.get("meal_id") or "",
current_meal_name=row.get("meal_name") or "",
current_meal_type=str(row.get("meal_type") or ""),
legacy_raw_iccid=legacy_iccid,
account_id=base_row.get("account_id") or "",
account_name=base_row.get("account_name") or "",
category=int(base_row.get("category") or 0),
category_name=category_names.get(int(base_row.get("category") or 0), ""),
agent_id=base_row.get("agent_id") or "",
agent_name=base_row.get("agent_name") or "",
msisdn=base_row.get("phone") or "",
virtual_no=virtual_nos.get(legacy_iccid, ""),
current_meal_id=meal_id,
current_meal_name=meal_row.get("meal_name") or "",
current_meal_type=str(meal_row.get("meal_type") or ""),
current_expire_date=expire_str,
current_series_id=row.get("series_id") or "",
current_series_name=row.get("series_name") or "",
current_series_id=series_id,
current_series_name=series_name,
)
timer.log("assemble", cards=len(metas), duplicates=len(duplicates))
return metas, duplicates
def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
def fetch_meal_meta(
conn,
meal_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyMealMeta]:
"""查套餐 + 系列元数据(供 scan_legacy 收集映射 legacy_* 字段)。"""
timer = _StageTimer("fetch_meal_meta")
id_list = sorted({m for m in meal_ids if m})
out: dict[str, LegacyMealMeta] = {}
if not id_list:
@@ -733,11 +910,15 @@ def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
LEFT JOIN tbl_set_meal_series sms ON sms.id = sm.meal_series_id
WHERE sm.id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(id_list):
for batch in _chunks(id_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
mid = row["meal_id"]
rows += 1
if not mid:
continue
out[mid] = LegacyMealMeta(
@@ -746,11 +927,17 @@ def fetch_meal_meta(conn, meal_ids: Iterable[str]) -> dict[str, LegacyMealMeta]:
series_id=row.get("series_id") or "",
series_name=row.get("series_name") or "",
)
timer.log("tbl_set_meal", meal_ids=len(id_list), batches=batches, rows=rows)
return out
def fetch_agent_balances(conn, agent_ids: Iterable[str]) -> dict[str, LegacyAgentBalance]:
def fetch_agent_balances(
conn,
agent_ids: Iterable[str],
batch_size: int = BATCH_SIZE,
) -> dict[str, LegacyAgentBalance]:
"""查代理主钱包余额(tbl_agent.balance_money,元单位)。"""
timer = _StageTimer("fetch_agent_balances")
agent_list = sorted({a for a in agent_ids if a})
out: dict[str, LegacyAgentBalance] = {}
if not agent_list:
@@ -761,12 +948,17 @@ def fetch_agent_balances(conn, agent_ids: Iterable[str]) -> dict[str, LegacyAgen
FROM tbl_agent
WHERE id IN %s
"""
rows = 0
batches = 0
with conn.cursor() as cur:
for batch in _chunks(agent_list):
for batch in _chunks(agent_list, batch_size):
batches += 1
cur.execute(sql, (tuple(batch),))
for row in cur.fetchall():
aid = row["id"]
rows += 1
if not aid:
continue
out[aid] = LegacyAgentBalance(agent_id=aid, balance_fen=_to_fen(row.get("balance_money")))
timer.log("tbl_agent", agents=len(agent_list), batches=batches, rows=rows)
return out

View File

@@ -139,6 +139,7 @@ class AgentMapping:
legacy_agent_id: str
legacy_agent_name: str
target_shop_code: Optional[str] # legacy_agent 模式下允许 null,表示该代理资产进平台库存
no_downtime: bool = False # 迁移时免停机:有生效套餐的卡直接以 network_status=1 导入
@dataclass
@@ -166,6 +167,11 @@ class Mapping:
return None
return self.series.get(str(legacy_series_id).strip())
def is_no_downtime_agent(self, legacy_agent_id: str) -> bool:
"""判断代理是否配置了免停机迁移。"""
agent = self.agents.get(str(legacy_agent_id or "").strip())
return agent.no_downtime if agent else False
def lookup_agent_shop_code(self, legacy_agent_id: str) -> Optional[str]:
"""按 agent_id 查 target_shop_code,空字符串视为 None(进平台库存)。"""
if not legacy_agent_id:
@@ -260,6 +266,7 @@ def load_mapping(config_dir: Path) -> Mapping:
legacy_agent_id=str(item["legacy_agent_id"]).strip(),
legacy_agent_name=str(item.get("legacy_agent_name") or ""),
target_shop_code=_opt_shop_code(item.get("target_shop_code"), "agents[].target_shop_code"),
no_downtime=bool(item.get("no_downtime", False)),
)
m.agents[a.legacy_agent_id] = a
@@ -536,6 +543,7 @@ def _append_agents(lines: list[str], mapping: Mapping) -> None:
f" - legacy_agent_id: {_yaml_scalar(item.legacy_agent_id)}",
f" legacy_agent_name: {_yaml_scalar(item.legacy_agent_name)}",
f" target_shop_code: {_yaml_scalar(item.target_shop_code)}",
f" no_downtime: {_yaml_scalar(item.no_downtime)}",
])

View File

@@ -621,6 +621,20 @@ def _write_iot_cards(
card_status = 2 if shop_code else 1
shop_id_expr = _shop_id_sub(shop_code)
# 免停机迁移:代理配置 no_downtime=true 且卡在奇成有当前生效正式套餐
# finalize 保证最后执行,此时 step2 套餐已落库,轮询系统不会在此之前触碰这张卡
is_no_downtime = bool(meta.current_meal_id) and mapping.is_no_downtime_agent(meta.agent_id)
if is_no_downtime:
activation_status = 1
real_name_status = 1 # 卡在线说明已实名,写 1 防止 EvaluateAndAct 因 not_realname 停机
network_status = 1
stop_reason_expr = "NULL"
else:
activation_status = 0
real_name_status = 0
network_status = 0
stop_reason_expr = _sql_str(_initial_polling_stop_reason(c))
f.write(
"INSERT INTO tb_iot_card (\n"
" iccid, iccid_19, iccid_20,\n"
@@ -636,9 +650,9 @@ def _write_iot_cards(
f" {_sql_str(c.iccid_full)}, {_sql_str(c.iccid_19)}, {iccid_20_expr},\n"
f" {carrier.target_carrier_id}, {_sql_str(carrier.target_carrier_type)}, {_sql_str(carrier.target_carrier_name)},\n"
f" {_sql_str(card_category)}, {_sql_str(mapping.migration_batch_no)},\n"
f" {card_status}, 0, 0, 0,\n"
f" {card_status}, {activation_status}, {real_name_status}, {network_status},\n"
f" 1, 1, {is_standalone},\n"
f" 'after_order', {_sql_str(_initial_polling_stop_reason(c))},\n"
f" 'after_order', {stop_reason_expr},\n"
f" {_sql_str(meta.msisdn) if meta.msisdn else 'NULL'}, "
f"{_sql_str(card_virtual_no) if card_virtual_no else 'NULL'}, "
f"{shop_id_expr}, {series_id_expr},\n"
@@ -1077,7 +1091,6 @@ def write_step2(
cards: list[CardRow],
devices: list[DeviceRow],
mapping: Mapping,
card_metas: dict[str, LegacyCardMeta],
usage_snapshots: dict[str, LegacyCardUsage],
package_lifecycles: dict[str, list[LegacyPackageLifecycle]],
commission_snapshots: dict[str, LegacyCommissionAccount],

View File

@@ -4,7 +4,7 @@
工作流:
1. 读 mapping.yaml(必须先跑过 scan_legacy.py)
2. 读 cards.csv / devices.csv
3. 连奇成查每张卡的元数据(category/agent_id/msisdn/virtual_no)
3. 连奇成查每张卡的元数据(category/agent_id/msisdn/virtual_no/当前生效套餐)
4. 应用 mapping 把 legacy_* 翻译成 target_*,自动决定归属
5. 生成 output/step1_*.sql + errors.csv + summary.txt
@@ -14,11 +14,13 @@
用法:
python3 migrate_assets.py [--config-dir config] [--resources-dir resources] [--output-dir output]
[--batch-size 1000]
"""
from __future__ import annotations
import argparse
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
@@ -31,6 +33,7 @@ def main() -> int:
parser.add_argument("--config-dir", default="config", help="yaml 配置目录")
parser.add_argument("--resources-dir", default="resources", help="CSV 资源目录")
parser.add_argument("--output-dir", default="output", help="SQL/CSV 输出目录")
parser.add_argument("--batch-size", type=int, default=legacy_query.BATCH_SIZE, help="老库批量查询大小")
args = parser.parse_args()
base = Path(__file__).resolve().parent
@@ -38,16 +41,26 @@ def main() -> int:
resources_dir = (base / args.resources_dir).resolve()
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
batch_size = args.batch_size
t0 = time.perf_counter()
mapping = mapping_loader.load_mapping(config_dir)
cards, devices, errors = csv_loader.load_assets(resources_dir, mapping)
print(f"[migrate] 加载 CSV: cards={len(cards)} devices={len(devices)} elapsed={time.perf_counter()-t0:.1f}s")
# 连奇成查卡元数据(运营商分类、agent_id、msisdn、virtual_no、当前生效套餐)
iccid_pairs = [(c.iccid_19, c.iccid_20, c.allow_iccid_19_lookup) for c in cards]
t1 = time.perf_counter()
dsn = mapping_loader.load_legacy_dsn(config_dir)
with legacy_query.connect_readonly(dsn) as conn:
card_metas, dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs)
print(f"[migrate] 连接老库: elapsed={time.perf_counter()-t1:.1f}s")
t2 = time.perf_counter()
card_metas, dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_card_meta: cards={len(card_metas)} duplicates={len(dup_iccids)} elapsed={time.perf_counter()-t2:.1f}s")
t3 = time.perf_counter()
sql_builder.write_step1(
output_dir,
cards=cards,
@@ -57,8 +70,10 @@ def main() -> int:
duplicate_iccids=dup_iccids,
errors=errors,
)
print(f"[migrate] SQL 生成: elapsed={time.perf_counter()-t3:.1f}s")
print(f"完成:卡 {len(cards)} 张,设备 {len(devices)} 台,异常 {len(errors)}")
total = time.perf_counter() - t0
print(f"完成:卡 {len(cards)} 张,设备 {len(devices)} 台,异常 {len(errors)} 行,总耗时 {total:.1f}s")
print(f"SQL 输出: {output_dir}")
return 0

View File

@@ -5,17 +5,19 @@
工作流:
1. 读 mapping.yaml + cards.csv
2. 连奇成查:卡元数据(card_meta) + 累计流量(card_usage) + 代理佣金/主钱包余额
2. 连奇成查:累计流量(card_usage) + 套餐生命周期 + 代理佣金/主钱包余额
3. 卡 → 当前生效套餐 → mapping.packages → target_package_id
4. 生成 step2_*.sql
用法:
python3 migrate_runtime.py [--config-dir config] [--resources-dir resources] [--output-dir output]
[--batch-size 1000]
"""
from __future__ import annotations
import argparse
import sys
import time
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
@@ -28,6 +30,7 @@ def main() -> int:
parser.add_argument("--config-dir", default="config", help="yaml 配置目录")
parser.add_argument("--resources-dir", default="resources", help="CSV 资源目录")
parser.add_argument("--output-dir", default="output", help="SQL/CSV 输出目录")
parser.add_argument("--batch-size", type=int, default=legacy_query.BATCH_SIZE, help="老库批量查询大小")
args = parser.parse_args()
base = Path(__file__).resolve().parent
@@ -35,9 +38,13 @@ def main() -> int:
resources_dir = (base / args.resources_dir).resolve()
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
batch_size = args.batch_size
t0 = time.perf_counter()
mapping = mapping_loader.load_mapping(config_dir)
cards, devices, errors = csv_loader.load_assets(resources_dir, mapping)
print(f"[migrate] 加载 CSV: cards={len(cards)} devices={len(devices)} elapsed={time.perf_counter()-t0:.1f}s")
if errors:
sql_builder.write_runtime_input_errors(output_dir, errors)
print(f"cards.csv/devices.csv 存在 {len(errors)} 个基础错误,已写入 {output_dir / 'errors.csv'}", file=sys.stderr)
@@ -46,27 +53,39 @@ def main() -> int:
iccid_pairs = [(c.iccid_19, c.iccid_20, c.allow_iccid_19_lookup) for c in cards]
agent_ids = [a.legacy_agent_id for a in mapping.agents.values() if (a.target_shop_code or "").strip()]
t1 = time.perf_counter()
dsn = mapping_loader.load_legacy_dsn(config_dir)
with legacy_query.connect_readonly(dsn) as conn:
card_metas, _dup_iccids = legacy_query.fetch_card_meta(conn, iccid_pairs)
usage_snapshots = legacy_query.fetch_card_usage(conn, iccid_pairs)
package_lifecycles = legacy_query.fetch_package_lifecycles(conn, iccid_pairs)
commission_snapshots = legacy_query.fetch_commission_accounts(conn, agent_ids) if agent_ids else {}
agent_balances = legacy_query.fetch_agent_balances(conn, agent_ids) if agent_ids else {}
print(f"[migrate] 连接老库: elapsed={time.perf_counter()-t1:.1f}s")
t2 = time.perf_counter()
usage_snapshots = legacy_query.fetch_card_usage(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_card_usage: cards={len(usage_snapshots)} elapsed={time.perf_counter()-t2:.1f}s")
t3 = time.perf_counter()
package_lifecycles = legacy_query.fetch_package_lifecycles(conn, iccid_pairs, batch_size=batch_size)
print(f"[migrate] fetch_package_lifecycles: cards={len(package_lifecycles)} elapsed={time.perf_counter()-t3:.1f}s")
t4 = time.perf_counter()
commission_snapshots = legacy_query.fetch_commission_accounts(conn, agent_ids, batch_size=batch_size) if agent_ids else {}
agent_balances = legacy_query.fetch_agent_balances(conn, agent_ids, batch_size=batch_size) if agent_ids else {}
print(f"[migrate] fetch_agent_data: agents={len(agent_ids)} elapsed={time.perf_counter()-t4:.1f}s")
t5 = time.perf_counter()
warnings = sql_builder.write_step2(
output_dir,
cards=cards,
devices=devices,
mapping=mapping,
card_metas=card_metas,
usage_snapshots=usage_snapshots,
package_lifecycles=package_lifecycles,
commission_snapshots=commission_snapshots,
agent_balances=agent_balances,
)
print(f"[migrate] SQL 生成: elapsed={time.perf_counter()-t5:.1f}s")
print(f"完成:卡 {len(cards)} 张,代理 {len(agent_ids)} 个,警告 {len(warnings)}")
total = time.perf_counter() - t0
print(f"完成:卡 {len(cards)} 张,代理 {len(agent_ids)} 个,警告 {len(warnings)} 条,总耗时 {total:.1f}s")
print(f"SQL 输出: {output_dir}")
return 0

View File

@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""卡删除脚本:读取 ICCID 列表,生成数据库删除 SQL 和 Redis 轮询清理命令。
用法:
python3 remove_cards.py --input <csv文件> [--output-dir output]
CSV 格式: 纯文本,每行一个 ICCID首行可有 "iccid" 表头(自动跳过)
输出:
output/remove_cards_db.sql -- 数据库软删 + 标识符硬删 SQL事务封装
output/remove_cards_redis_gen.sql -- 在数据库执行后把输出通过 redis-cli 清理轮询队列
"""
from __future__ import annotations
import argparse
import sys
from datetime import datetime
from pathlib import Path
def load_iccids(csv_path: Path) -> list[str]:
"""读取 CSV 文件中的 ICCID 列表,自动跳过空行和表头。"""
iccids: list[str] = []
with csv_path.open(encoding="utf-8") as f:
for lineno, raw in enumerate(f, 1):
line = raw.strip()
if not line:
continue
# 跳过纯字母表头(如 "iccid"
if lineno == 1 and not line.isdigit() and not line.startswith("8"):
print(f"[remove] 跳过表头: {line!r}", file=sys.stderr)
continue
iccids.append(line)
return iccids
def sql_in_list(iccids: list[str]) -> str:
"""生成 SQL IN 列表字符串,如 '898...', '898...'"""
return ", ".join(f"'{i}'" for i in iccids)
def generate_db_sql(iccids: list[str], generated_at: str) -> str:
in_list = sql_in_list(iccids)
count = len(iccids)
return f"""\
-- ============================================================
-- 迁移卡删除脚本 (数据库部分)
-- 生成时间: {generated_at}
-- 卡数量 : {count}
-- 执行前请先用 remove_cards_redis_gen.sql 生成 Redis 清理命令并执行
-- 建议执行: psql ... -1 -f <此文件>
-- ============================================================
BEGIN;
-- 临时表:本次要删除的卡
CREATE TEMP TABLE tmp_remove_cards AS
SELECT id, iccid
FROM tb_iot_card
WHERE iccid IN ({in_list});
DO $$ BEGIN
RAISE NOTICE '本次将删除 % 张卡(共输入 {count} 个 ICCID', (SELECT COUNT(*) FROM tmp_remove_cards);
END $$;
-- 1. 删除套餐使用记录
DELETE FROM tb_package_usage
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 2. 删除订单
DELETE FROM tb_order
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 3. 删除分配记录
DELETE FROM tb_asset_allocation_record
WHERE asset_type = 'iot_card'
AND asset_id IN (SELECT id FROM tmp_remove_cards);
-- 4. 删除资产钱包
DELETE FROM tb_asset_wallet
WHERE resource_type = 'iot_card'
AND resource_id IN (SELECT id FROM tmp_remove_cards);
-- 5. 删除设备绑卡关系
DELETE FROM tb_device_sim_binding
WHERE iot_card_id IN (SELECT id FROM tmp_remove_cards);
-- 6. 删除资产标识符
DELETE FROM tb_asset_identifier
WHERE asset_type = 'iot_card'
AND asset_id IN (SELECT id FROM tmp_remove_cards);
-- 7. 删除卡本体(最后执行)
DELETE FROM tb_iot_card
WHERE id IN (SELECT id FROM tmp_remove_cards);
COMMIT;
"""
def generate_redis_gen_sql(iccids: list[str], generated_at: str) -> str:
in_list = sql_in_list(iccids)
count = len(iccids)
return f"""\
-- ============================================================
-- 轮询队列清理命令生成 SQL
-- 生成时间: {generated_at}
-- 卡数量 : {count}
-- 执行方式:
-- psql ... -t -A -c "$(cat remove_cards_redis_gen.sql)" | redis-cli --pipe
-- 或:
-- psql ... -t -A -f remove_cards_redis_gen.sql | redis-cli
-- ============================================================
WITH cards AS (
SELECT id FROM tb_iot_card
WHERE iccid IN ({in_list})
),
task_types(task_type) AS (
VALUES
('polling:realname'),
('polling:carddata'),
('polling:package'),
('polling:protect'),
('polling:card_status')
)
SELECT format('ZREM polling:shard:%s:queue:%s %s', id % 16, task_type, id)
FROM cards CROSS JOIN task_types
UNION ALL
SELECT format('DEL polling:card:%s', id)
FROM cards
UNION ALL
SELECT format('LREM polling:manual:%s 0 %s', task_type, id)
FROM cards CROSS JOIN task_types
UNION ALL
SELECT format('SREM polling:manual:dedupe:%s %s', task_type, id)
FROM cards CROSS JOIN task_types;
"""
def main() -> int:
parser = argparse.ArgumentParser(description="迁移卡删除脚本:生成删除 SQL 和 Redis 清理命令")
parser.add_argument("--input", required=True, help="ICCID 列表 CSV 文件路径")
parser.add_argument("--output-dir", default="output", help="输出目录(默认 output")
args = parser.parse_args()
base = Path(__file__).resolve().parent
input_path = Path(args.input)
if not input_path.is_absolute():
input_path = base / input_path
output_dir = (base / args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
if not input_path.exists():
print(f"[remove] 错误: 找不到输入文件 {input_path}", file=sys.stderr)
return 1
iccids = load_iccids(input_path)
if not iccids:
print("[remove] 错误: CSV 中没有找到有效的 ICCID", file=sys.stderr)
return 1
print(f"[remove] 读取到 {len(iccids)} 个 ICCID开始生成 SQL...")
generated_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
db_sql_path = output_dir / "remove_cards_db.sql"
redis_sql_path = output_dir / "remove_cards_redis_gen.sql"
db_sql_path.write_text(generate_db_sql(iccids, generated_at), encoding="utf-8")
redis_sql_path.write_text(generate_redis_gen_sql(iccids, generated_at), encoding="utf-8")
print(f"[remove] 生成完成:")
print(f" 数据库删除 SQL : {db_sql_path}")
print(f" Redis 清理 SQL : {redis_sql_path}")
print()
print("执行步骤:")
print(" 1. 先执行 Redis 清理(在删除卡数据之前,以便 SQL 还能查到卡 ID:")
print(f" psql <连接参数> -t -A -f {redis_sql_path} | redis-cli")
print(" 2. 再执行数据库删除:")
print(f" psql <连接参数> -1 -f {db_sql_path}")
return 0
if __name__ == "__main__":
sys.exit(main())