From 99e03f4f47df3d4c2b1a61180891438ca159a190 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Wed, 12 Aug 2026 21:52:11 -0700 Subject: [PATCH 01/19] docs(openspec): propose the agent command and the Vale authoring path `add-vale-rule-engine` shipped a chooser without a destination. `engine-selection` teaches an agent to conclude a rule belongs to `vale`, and then there is nowhere to go: `static.txt` is ast-grep authoring, and its only mention of Vale tells the reader to confirm `sg` was right. The `runtime` answer dead-ends the same way. The exclusion was recorded as a deliberate non-goal at the time, which was defensible while nothing chose Vale -- the chooser is what makes the gap reachable. The same work exposed that the surface is shaped for a human. `help` names the command after a human's reason for typing it, and longform addressing makes an agent know both the words and their order. Single hyphenated tokens read as literal strings to copy rather than phrases to paraphrase, which is the failure this surface cannot afford. Proposes: `help` -> `agent`, flattened single-token topics, `static` -> `create-sg-rule` and `existing` -> `create-legacy-rule`, new `create-vale-rule` and `create-runtime-rule`, `route` merged with `engine-selection` into one front door, and a section-less scaffold. Two decisions carry the most weight. Merging `route` and `engine-selection` removes a second fetch and a handoff to answer one question, but it deletes a topic the platform generator consumes -- so the criterion distributes to the destinations, each `create-*-rule` stating the evidence that makes its engine right. That is what keeps it exportable, since a consumer with no `route` step and no `detect --json` could never use a chooser anyway. And the section-less scaffold ships only paired with surfacing Vale's stderr on a zero-exit run: with no section to copy, the likely first edit is a rule assignment at top level, which Vale ignores and reports on stderr, which we currently discard -- reintroducing the silent disable one level up. Pre-1.0, every backwards-incompatible item here is a MINOR bump. Delivery shape: single PR stacked on #100. A half-renamed command surface is not shippable, since recipes cross-reference each other by literal command string. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../.openspec.yaml | 2 + .../design.md | 103 ++++++++++++++++++ .../proposal.md | 49 +++++++++ .../specs/cli-agent-authoring/spec.md | 63 +++++++++++ .../specs/cli-help/spec.md | 60 ++++++++++ .../specs/cli-knowledge-prompts/spec.md | 39 +++++++ .../specs/cli-rule-routing/spec.md | 88 +++++++++++++++ .../specs/cli-vale-rule-engine/spec.md | 39 +++++++ .../agent-command-and-vale-authoring/tasks.md | 45 ++++++++ 9 files changed, 488 insertions(+) create mode 100644 openspec/changes/agent-command-and-vale-authoring/.openspec.yaml create mode 100644 openspec/changes/agent-command-and-vale-authoring/design.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/proposal.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md create mode 100644 openspec/changes/agent-command-and-vale-authoring/tasks.md diff --git a/openspec/changes/agent-command-and-vale-authoring/.openspec.yaml b/openspec/changes/agent-command-and-vale-authoring/.openspec.yaml new file mode 100644 index 0000000..b6b2d1f --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-13 diff --git a/openspec/changes/agent-command-and-vale-authoring/design.md b/openspec/changes/agent-command-and-vale-authoring/design.md new file mode 100644 index 0000000..1b06336 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/design.md @@ -0,0 +1,103 @@ +## Context + +The CLI's `help/*.txt` recipes are the knowledge surface agents read. They are addressed longform (`positionals.join("-")` in `commands/help.ts`), exported in part through `@taskless/cli/prompts` for the platform generator, and cross-referenced from each other by literal command string — ~306 occurrences of `taskless help` across 77 files. + +`add-vale-rule-engine` added `engine-selection`, which decides between `sg`, `vale`, and `runtime`. Two of those three answers have no authoring procedure. That change recorded Vale authoring as an explicit non-goal, which was defensible when nothing chose Vale; the chooser makes it reachable. + +Separately, the scaffolded `.vale.ini` opens an unscoped `[*]`. A whole-project check under it lints build output and, until #100, `.taskless/` itself. + +## Goals / Non-Goals + +**Goals:** + +- Every answer `route` can produce leads to a procedure that exists. +- One fetch from request to destination, returning a command an agent can run verbatim. +- A topic vocabulary that reads as literal tokens rather than paraphrasable phrases. +- A scaffold that lints nothing until someone scopes it deliberately. + +**Non-Goals:** + +- **A `.vale.ini` writer.** The agent authors the section, exactly as it authors `sgconfig.yml` rule entries today. Construction moves to the downstream generator, consistent with `add-vale-rule-engine/design.md:107`. +- Rule _generation_ for Vale via the service. `remote` remains the service path; this change gives the local path a destination. +- Renaming the `cli-help` capability file. The command renames; the spec keeps its name, so this change does not also move spec files (see D5). +- Restricting Vale's feature set, or deciding build-output exclusion (tracked separately in #101). + +## Decisions + +### D1 — `route` and `engine-selection` merge into one topic + +There is one decision, made once. `route` absorbs the engine reasoning and returns one of the four `create-*-rule` topics. `engine-selection` ceases to exist as a separate topic. + +This reverses the scoping `engine-selection` asserts today — that route decides destination, the topic decides engine, and locally the two compose. The separation is clean on paper and expensive in practice: it costs an agent two fetches and a correct handoff between them to answer one question, and the handoff is where an agent drops context. Worse, the two decisions are not independent in the direction the split assumes; "author this locally" and "which engine can express it" are answered from the same evidence, so splitting them means reading the same signals twice. + +**Where the reasoning goes for consumers outside the CLI.** The platform generator consumes `engine-selection` through `TOPICS` and has no `route` step, so merging cannot simply delete what it reads. Exporting `route` instead is wrong — it is built on `taskless detect --json` and local authoring, neither of which a Worker can do. + +The reasoning therefore **distributes to the destinations**: each `create-*-rule` recipe states the evidence that makes its engine the right one, and `route` applies those same tests to dispatch. A consumer choosing between `create-sg-rule` and `create-vale-rule` reads the criterion at the destination rather than fetching a chooser first. This keeps the exported surface honest — every exported topic is one a service-side consumer can actually act on — and it removes the class of drift where a chooser and its destinations disagree about when each applies. + +_Alternative rejected:_ keep `engine-selection` as a third exported topic that `route` also applies. Two statements of the same criterion, guaranteed to drift, and it preserves the second fetch for exactly the consumer we were trying to simplify. + +_Alternative rejected:_ export `route`. It is local-only by construction. + +### D2 — Verb-noun names, single token, no aliases + +`create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, `create-legacy-rule`. Not everything needs the noun — `route` stays `route`. + +Hyphenated single tokens are the point rather than a side effect. A multi-word phrase invites an agent to paraphrase or reorder; a hyphenated token reads as a literal string to copy. This is the same reason the resolution stops joining positionals: with one token there is no order to get wrong. + +`static` → `create-sg-rule` also removes a leak. "Static" is a trust tier, and `engine-selection` is explicit that tier and engine are different axes; naming the ast-grep authoring topic after the tier taught the confusion the other topic exists to correct. + +_Alternative rejected:_ `static-sg` / `static-vale`. Preserves the tier leak and does not match what `route` decides. + +### D3 — Break `TOPICS`, no deprecation window + +`TOPICS` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]`. `engine-selection` leaves the export because it stops existing (D1); the criterion it carried is now stated by the destinations themselves. + +The package is pre-1.0, so this ships **MINOR**. That is what the leading zero means, and it holds for every backwards-incompatible item in this change. An alias would have to be carried by the type union (`PromptTopic`), the `PROMPTS` map, and the disjointness test, and would be dead the moment the generator updates. + +The real exposure is not the rename but the **deploy skew**: the generator is a separate deploy consuming a published package, so it breaks on upgrade rather than at our build time. Mitigated by the changeset, not by code. + +_Alternative rejected:_ export both names for one release. Doubles the exported surface to protect a single known consumer that we control. + +### D4 — Section-less scaffold, and stderr notices as its precondition + +The scaffolded `.vale.ini` carries `StylesPath` and `MinAlertLevel` and no section. Measured: Vale runs clean and reports `{}`. + +This is only safe **with** stderr surfacing, and the two ship together. With no section to copy, the natural first edit is `rules. = YES` at top level, which Vale reports as `W101 '' isn't a core option; Vale is ignoring it` — on stderr, with exit 0 and valid `{}` on stdout. `runVale` reads stderr only on a non-zero exit, so today that diagnostic is discarded and the user gets: rule authored, `verify` passes, `check` silent. That is the exact failure the Vale work exists to eliminate, and shipping the scaffold change alone would reintroduce it one level up. + +_Alternative rejected:_ scaffold `[*.md]`. A user's first rule works immediately, but the default silently decides scope for them, and markdown is a guess about what a repo's prose is. + +### D5 — The `cli-help` capability keeps its name + +The command becomes `agent`; the spec file stays `openspec/specs/cli-help/spec.md`. + +Renaming a capability means moving a spec directory and rewriting every cross-reference to it in the same change that already renames a command and four topics. The capability's subject — the CLI's agent-knowledge surface — is unchanged; only its command name moves. Worth doing later on its own, and worth not doing here. + +## Risks / Trade-offs + +- **Generator breaks on CLI upgrade, not at build time** → `TOPICS` is consumed across a deploy boundary. The changeset must name the rename explicitly, and the generator's update is a coordinated follow-up rather than an assumption. +- **306 mechanical edits invite a missed one** → a stale `taskless help X` in a recipe is invisible until an agent runs it and gets nothing. Mitigated by an assertion that no shipped recipe contains the string `taskless help`, which is cheap and total. +- **Absorbing engine choice into `route` makes `route` longer** → it now carries the reasoning that justified a separate topic. If it grows past being readable in one pass, the split was load-bearing after all and should come back as a fetch. +- **A section-less scaffold means a fresh project's first Vale rule does nothing until scoped** → intended, and the reason `create-vale-rule` must teach the section rather than assume it. The stderr notice is what makes the failure legible instead of silent. +- **Telemetry vocabulary changes** → `cli_help` topic values change wholesale; anything keyed on `static` goes quiet rather than erroring. Worth naming before it is diagnosed as a traffic drop. + +## Migration Plan + +No user data or on-disk state migrates. Existing projects keep whatever `.vale.ini` they have — the scaffold change affects new projects only, and `0004` is unreleased, so no project has the old scaffold in the field. + +The rename is a hard cutover in one PR: recipes cross-reference each other by literal command, so a partial rename produces recipes pointing at commands that do not exist. + +### D6 — There is no `create-remote-rule`; `create-runtime-rule` covers the gated path + +`route` names four destinations, all engine-shaped. "Remote" is not a fifth engine — it describes who generates, which is a different question, and the one place it bites is the same place `create-runtime-rule` already has to speak: the user is logged out. + +So `create-runtime-rule` owns the gated story end to end — why executing code requires login, reconciliation, and signing, and what to do when the user has none of them. A logged-out user meets one topic explaining one gate, rather than being routed to a topic about remoteness that then explains authentication. + +**What this leaves unresolved, deliberately.** `remote.txt` and `rule-create.txt` today serve a real and different flow: the service generating an _ast-grep_ rule when local authoring cannot. That is escalation, not a destination — `route` is already specified as biased local with the service as last resort — so it survives as a fallback inside `route` rather than as a peer of the four. Whether those two recipes keep their names, merge, or fold into `create-sg-rule`'s failure path is not settled here. + +_Alternative rejected:_ a fifth `create-remote-rule` destination. Puts a non-engine on an engine-shaped list, and splits the logged-out explanation across two topics. + +## Open Questions + +- `remote.txt` and `rule-create.txt` are no longer named by `route`, but still implement service generation. Do they remain fetchable topics, fold into `create-sg-rule`'s escalation path, or merge with each other? Not settled by D6, and worth deciding before the sweep in task group 3 rewrites their cross-references. +- Should `create-runtime-rule` explain the login requirement itself, or defer to `auth`? Stated as the former; worth confirming it does not duplicate `auth`. +- Does the distributed engine criterion (D1) belong in each recipe's Preconditions or as a named section? Affects whether a consumer can extract it mechanically. diff --git a/openspec/changes/agent-command-and-vale-authoring/proposal.md b/openspec/changes/agent-command-and-vale-authoring/proposal.md new file mode 100644 index 0000000..b6e209e --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/proposal.md @@ -0,0 +1,49 @@ +## Why + +`add-vale-rule-engine` shipped a chooser without a destination. `engine-selection` teaches an agent to decide a rule belongs to `vale`, and then there is nowhere to go: `static.txt` is 76 lines of ast-grep authoring, and its only mention of Vale is a See Also line telling the reader to _confirm_ `sg` was right. An agent that follows the procedure correctly and lands on `vale` dead-ends, and so does one that lands on `runtime`. That was recorded as a deliberate non-goal at the time (`add-vale-rule-engine/design.md:16` excludes "generating Vale rules and authoring the committed `.vale.ini`"), but the chooser is what makes the gap reachable, and it is now shipped. + +The same work exposed that the surface an agent reads is shaped for a human. `taskless help` names the command after a human's reason for typing it; agents are not asking for help, they are fetching a procedure. And the surface is addressed longform — `taskless help rule create` resolves by joining positionals — so an agent must know both the words and their order. Hyphenated single tokens read as literal strings an agent copies rather than a phrase it might paraphrase, which is the failure this surface cannot afford. + +## What Changes + +- **BREAKING** — `taskless help ` becomes `taskless agent `. The command is named for who reads it. +- **BREAKING** — topic addressing flattens to a single token. `taskless help rule create` becomes `taskless agent create-rule`; the `positionals.join("-")` resolution is removed rather than generalized. +- **BREAKING** — authoring topics are renamed to verb-noun, matching what `route` decides: + - `static` → `create-sg-rule` + - `existing` → `create-legacy-rule` + - new `create-vale-rule` + - new `create-runtime-rule` +- **BREAKING** — `TOPICS` in `@taskless/cli/prompts` renames with them. `["static", "engine-selection"]` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]`. No alias is kept. Pre-1.0, a backwards-incompatible change is a **MINOR** bump. +- `route` becomes the single front door, returning a concrete next command rather than a category. Its decision set is the four `create-*-rule` topics; service generation stays an escalation for when local authoring cannot express the rule, not a peer destination. +- **BREAKING** — `engine-selection` merges into `route` and stops existing as a topic. Its criterion distributes: `route` applies it to dispatch, and each `create-*-rule` recipe states the evidence that makes its own engine right. That is what keeps it exportable — a consumer outside the CLI has no `route` step and cannot run `taskless detect --json`, so a chooser topic was unusable to it anyway. +- `create-vale-rule` covers what no topic covers today: authoring a Vale style file under `vale/rules/`, scoping it with a `.vale.ini` section, and writing `pass/`/`fail` fixtures. Consistent with `create-sg-rule`, the agent writes these files; no CLI writer is introduced. +- The scaffolded `.vale.ini` ships **no section**, so a fresh project lints nothing until a user scopes something deliberately. `create-vale-rule` teaches writing that first section. +- Vale's stderr diagnostics on a successful run surface as notices. This is required by the change above, not incidental: with no section to copy, the likely first mistake is a rule assignment at top level, which Vale reports as `W101 ... is ignoring it` on stderr and which today is discarded — reproducing the silent-disable class the Vale work exists to eliminate. + +## Capabilities + +### New Capabilities + +- `cli-agent-authoring`: the four `create-*-rule` procedures — what each engine's authored artifacts are, where they live, and what makes one complete. Covers the Vale authoring path that has no home today. + +### Modified Capabilities + +- `cli-help`: the command renames to `agent` and topic addressing flattens to a single token. Longform resolution is removed. +- `cli-rule-routing`: `route` dispatches to a concrete `create-*-rule` topic rather than a category, absorbing the engine decision it previously deferred to `engine-selection`. +- `cli-knowledge-prompts`: `TOPICS` renames, gains the Vale and runtime authoring topics, and loses `engine-selection`; pre-1.0 breaking changes are restated as MINOR. +- `cli-vale-rule-engine`: the scaffolded config carries no section, and Vale's stderr diagnostics on a zero-exit run become notices. + +## Impact + +- **`packages/cli/src/commands/help.ts`** — renamed, positional-join resolution removed. +- **`packages/cli/src/help/*.txt`** — two renames, two new files, and ~306 cross-references across 77 files that name `taskless help`. +- **`packages/cli/src/prompts/index.ts`** — `TOPICS`/`INTERNAL_TOPICS` membership and the `PromptTopic` union. +- **`@taskless/cli/prompts`** — published, typed export. The platform generator consumes `TOPICS` and deploys separately from the CLI, so it breaks on upgrade rather than at build time. Flagged in the changeset. +- **`packages/cli/src/filesystem/migrations/0004-vale-engine.ts`** — `VALE_CONFIG_CONTENT` drops its `[*]` section. +- **`packages/cli/src/rules/vale/run.ts`** — stderr captured on a zero-exit run and returned as a notice. +- **`skills/taskless/SKILL.md`** — the one skill naming `taskless help`. +- **Telemetry** — `cli_help` events carry a `topic` whose vocabulary changes; dashboards keyed on `static` go quiet. + +## Delivery Shape + +**Single PR**, stacked on #100. The rename is mechanical but total: a half-renamed command surface is not a shippable intermediate state, and splitting the topic renames from the command rename would leave cross-references pointing at commands that do not exist yet. Reviewable because the diff is overwhelmingly one substitution repeated, with four files of genuinely new prose. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md new file mode 100644 index 0000000..1aad0d1 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md @@ -0,0 +1,63 @@ +## ADDED Requirements + +### Requirement: Every engine a rule can be routed to has an authoring recipe + +The CLI SHALL provide an authoring recipe for each engine `route` can name: `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule`, alongside `create-legacy-rule` for a linter the repository already uses. + +A decision procedure that can produce an answer with no destination is incomplete. Engine selection can conclude `vale` or `runtime`, and before this change neither had a procedure, so an agent that reasoned correctly arrived nowhere. + +#### Scenario: Each engine choice reaches a procedure + +- **WHEN** engine selection concludes `sg`, `vale`, or `runtime` +- **THEN** a recipe exists that authors a rule for that engine + +#### Scenario: A legacy destination exists for repositories with their own linter + +- **WHEN** the repository already runs a linter that can express the rule +- **THEN** `create-legacy-rule` SHALL author it in that tool's own dialect + +### Requirement: The Vale authoring recipe covers rule, scope, and fixtures + +The `create-vale-rule` recipe SHALL instruct the agent to produce three artifacts, and SHALL state that a rule is incomplete without all three: + +1. A Vale style file under `.taskless/vale/rules/.yml`. +2. A section in the committed `.taskless/vale/.vale.ini` scoping which files the rule applies to, enabling it as `rules. = YES`. +3. `pass/` and `fail/` fixture documents under `.taskless/vale/rule-tests//`. + +The recipe SHALL state that the scaffolded config carries no section, so the first rule authored in a project also authors the first scope. + +#### Scenario: Authoring produces all three artifacts + +- **WHEN** the agent follows `create-vale-rule` +- **THEN** it writes the style file, a scoping section enabling the rule, and both fixture buckets + +#### Scenario: The recipe teaches the first section + +- **WHEN** a project's `.vale.ini` has no section yet +- **THEN** the recipe SHALL direct the agent to add one scoped to the files the rule is about, rather than assuming a section exists + +#### Scenario: An unscoped rule is not silently accepted + +- **WHEN** the agent enables a rule without placing it inside a section +- **THEN** the recipe SHALL identify this as incomplete, because Vale ignores a rule assignment outside a section + +### Requirement: Authoring recipes write files rather than invoking a writer + +The `create-*-rule` recipes SHALL instruct the agent to write the rule, its configuration, and its fixtures directly. The CLI SHALL NOT provide a command that generates a Vale style file or edits `.vale.ini` on the agent's behalf. + +This matches how ast-grep rules are authored today: the agent writes the rule and its config entry, and construction belongs to the downstream generator rather than to the CLI. + +#### Scenario: No CLI writer for Vale configuration + +- **WHEN** an agent authors a Vale rule +- **THEN** it edits `.vale.ini` itself +- **AND** the CLI SHALL NOT offer a subcommand that performs that edit + +### Requirement: The runtime authoring recipe states why it is gated + +The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, and SHALL state this as a property of executing code rather than as a property of the engine's capability. + +#### Scenario: The gate is explained where it is encountered + +- **WHEN** an agent follows `create-runtime-rule` +- **THEN** the recipe SHALL state why the runtime tier is gated when the static tiers are not diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md new file mode 100644 index 0000000..72a1d57 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md @@ -0,0 +1,60 @@ +## MODIFIED Requirements + +### Requirement: Help subcommand displays rich help text for commands + +The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic AND an optional `--anonymous` boolean flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching help text file embedded at build time using the following resolution order: + +1. If `--anonymous` is set AND `.anonymous.txt` exists in the embedded map, return that file. +2. Otherwise, return `.txt`. +3. If neither exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index. + +When no positional argument is provided, the subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries. + +The subcommand is named for its reader. It serves agents fetching a procedure, not humans asking for help, and single-token addressing exists so a topic name is a literal string an agent copies rather than a phrase it can reorder or paraphrase. + +#### Scenario: Agent subcommand for a topic returns the recipe + +- **WHEN** a user runs `taskless agent check` +- **THEN** the CLI SHALL print the contents of `check.txt` to stdout + +#### Scenario: Multi-word topic paths are not resolved + +- **WHEN** a user runs `taskless agent rule create` +- **THEN** the CLI SHALL NOT look up `rule-create.txt` by joining the positionals +- **AND** it SHALL exit non-zero rather than guessing a topic + +#### Scenario: Formerly nested topics are addressed by one token + +- **WHEN** a user runs `taskless agent create-rule` +- **THEN** the CLI SHALL look up `create-rule.txt` and print its contents + +#### Scenario: The former command name is gone + +- **WHEN** a user runs `taskless help check` +- **THEN** the CLI SHALL NOT print recipe text for `check` + +### Requirement: Routing recipes reference engine selection + +The `route` recipe SHALL apply the engine-selection reasoning directly and name a concrete authoring topic, rather than referring the reader onward to select an engine. The `create-sg-rule` and `create-vale-rule` recipes SHALL reference `engine-selection` so an agent that arrives at one directly can confirm the engine is right for the rule in hand. + +#### Scenario: Route names a destination without a second fetch + +- **WHEN** an agent follows `route` +- **THEN** the recipe SHALL name one `create-*-rule` topic or `remote` +- **AND** it SHALL NOT require fetching `engine-selection` first to do so + +#### Scenario: Authoring recipes cross-reference engine selection + +- **WHEN** an agent reads `create-sg-rule` or `create-vale-rule` +- **THEN** the recipe SHALL reference `taskless agent engine-selection` as the check that the engine matches the rule + +## ADDED Requirements + +### Requirement: Shipped recipes name only commands that exist + +No embedded recipe SHALL contain the string `taskless help`. Recipes cross-reference each other by literal command string, so a stale reference is invisible until an agent runs it and receives nothing. + +#### Scenario: No recipe references the removed command + +- **WHEN** the embedded recipe set is inspected +- **THEN** no recipe SHALL contain `taskless help` diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md new file mode 100644 index 0000000..fce97a1 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md @@ -0,0 +1,39 @@ +## MODIFIED Requirements + +### Requirement: Topic names and accessor shape are stable public API + +The set of `PromptTopic` names, the `getPrompt`/`PROMPTS` shape, and the existing fields of `PromptOptions` SHALL be treated as public API; recipe _text_ MAY change freely. + +The package is pre-1.0, so a backwards-incompatible change to that surface SHALL be released as a **MINOR** bump. This is what the leading zero means, and it applies to renaming a topic, removing one, or changing the accessor signature. + +What the requirement actually protects is not the version number but the notice. `TOPICS` is consumed across a deploy boundary, so a downstream consumer breaks when it upgrades rather than when this package builds, and the version alone cannot warn anyone. A breaking change SHALL therefore name the removed or renamed topics explicitly in its changeset. + +#### Scenario: Renaming or removing a topic + +- **WHEN** a topic is removed or renamed, or the accessor signature changes +- **THEN** it SHALL be released as a MINOR bump +- **AND** the changeset SHALL name the removed or renamed topics +- **AND** a recipe text edit SHALL require neither + +#### Scenario: Adding an option + +- **WHEN** a new optional field is added to `PromptOptions` +- **THEN** it SHALL NOT require more than a PATCH bump, since existing call sites keep their behavior + +## ADDED Requirements + +### Requirement: Exported topics cover both static-tier authoring paths + +`TOPICS` SHALL export the authoring recipe for each static-tier engine — `create-sg-rule` and `create-vale-rule` — alongside `engine-selection`. + +A consumer that can decide a rule belongs to `vale` must be able to reach the procedure for authoring one. Exporting the chooser without the Vale destination reproduces, for the platform generator, the dead end this change removes from the CLI. + +#### Scenario: Both authoring paths are reachable from the export + +- **WHEN** a consumer imports `TOPICS` +- **THEN** it SHALL contain `create-sg-rule`, `create-vale-rule`, and `engine-selection` + +#### Scenario: The exported set follows the rename + +- **WHEN** a consumer imports `TOPICS` +- **THEN** it SHALL NOT contain `static`, which no longer names a recipe diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md new file mode 100644 index 0000000..0b18fd9 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md @@ -0,0 +1,88 @@ +## MODIFIED Requirements + +### Requirement: Route is the local authoring classifier + +The CLI SHALL provide a `route` help recipe that instructs the agent to classify a rule-authoring request into one of four destinations — `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, or `create-runtime-rule` — using `taskless detect --json` signals plus the user's intent. The `route` recipe SHALL be biased to stay local, treating service generation as an escalation available when local authoring cannot express the rule rather than as a peer destination. + +`route` SHALL decide the engine as part of this classification rather than deferring it to a separate topic. There is one decision, made from one reading of the evidence: whether a rule is expressible locally and which engine can express it are answered from the same signals, so splitting them costs a second fetch and a handoff without adding information. + +Each destination SHALL be a topic an agent can fetch by name, so classifying produces a command to run rather than a category to interpret. + +#### Scenario: Route fetches detection before classifying + +- **WHEN** the agent fetches the `route` recipe to author a rule +- **THEN** the recipe SHALL direct the agent to run `taskless detect --json` and + use its signals as input to the classification + +#### Scenario: Route classifies into one of four destinations + +- **WHEN** the agent follows `route` +- **THEN** it SHALL select exactly one of `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, or `create-runtime-rule` +- **AND** it SHALL fetch the corresponding recipe to perform the authoring + +#### Scenario: Every destination resolves to a recipe + +- **WHEN** any destination `route` can name is fetched +- **THEN** a recipe of that exact name SHALL exist + +#### Scenario: The engine is decided without a second fetch + +- **WHEN** the agent follows `route` +- **THEN** it SHALL arrive at an engine-specific recipe without fetching a separate engine-selection topic + +### Requirement: Static recipe authors a verified local ast-grep rule + +The CLI SHALL provide a `create-sg-rule` help recipe that instructs the agent to author a +local ast-grep rule on-device, without calling the Taskless service, and to +verify it against the user's success and failure cases before reporting success. +The recipe SHALL produce the canonical on-disk rule shape and paths used by remote +generation so that `check`, `improve`, and `verify` see a single dialect. + +The recipe SHALL be named for the artifact it produces rather than for a trust tier. "Static" describes when a rule runs, which is a different axis from which engine enforces it, and naming the ast-grep authoring path after the tier taught the conflation that engine selection exists to correct. + +#### Scenario: Local authoring without the service + +- **WHEN** the agent follows `create-sg-rule` +- **THEN** it SHALL write the rule on-device without requiring login or the + Taskless API + +### Requirement: Trust tier is not an engine-selection input + +Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to the engine choice: `sg` and `vale` are both static-tier, and only `runtime` carries those concerns, so trust tier is a distinct axis from which engine can express a rule. + +#### Scenario: Trust tier is not an engine-selection input + +- **WHEN** the reasoning distinguishes `sg` from `vale` +- **THEN** it does so on the prose-versus-structure axis, not on any auth, reconcile, or signing property, since both are static-tier + +## ADDED Requirements + +### Requirement: Engine reasoning lives in route and in each destination + +The engine criterion SHALL be stated in `route`, which applies it to dispatch, and in each `create-*-rule` recipe, which states the evidence that makes its own engine the right one. It SHALL NOT be stated in a separate chooser topic. + +Distributing it this way is what lets the reasoning stay exportable. A consumer outside the CLI has no `route` step and cannot run `taskless detect --json`, so a chooser topic is unusable to it; a destination that states its own criterion is usable by anything that can read one recipe. It also removes the drift class where a chooser and its destinations disagree about when each applies. + +#### Scenario: A destination states when it applies + +- **WHEN** an agent or consumer reads `create-sg-rule` or `create-vale-rule` +- **THEN** the recipe SHALL state the evidence that makes that engine the right one for a rule + +#### Scenario: No separate chooser topic exists + +- **WHEN** the embedded recipe set is inspected +- **THEN** there SHALL be no topic whose only purpose is selecting among engines + +## REMOVED Requirements + +### Requirement: Engine selection is a separate axis from authoring destination + +**Reason**: The separation cost an agent two fetches and a handoff to answer one question. Whether a rule is expressible locally and which engine can express it are answered from the same evidence, so reading it twice added a failure point without adding information. + +**Migration**: The engine criterion moves into `route` and into each `create-*-rule` recipe (see "Engine reasoning lives in route and in each destination"). The one part of this requirement that was not about the split — that trust tier is a distinct axis — is retained as its own requirement above. Consumers that fetched `engine-selection` read the destination recipes instead, which is the surface `TOPICS` now exports. + +### Requirement: An engine-selection topic states which engine can enforce a rule + +**Reason**: The topic it required no longer exists as a separate recipe. + +**Migration**: Its content — the three engine definitions, evidence-before-answer, and the boundary cases — moves into `route` and the `create-*-rule` recipes. The requirements that constrained the reasoning itself ("Available code context outranks the phrasing of the request", "Ambiguity resolves to an engine known to be available") remain in force and now bind `route` and the destination recipes rather than a standalone topic. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md new file mode 100644 index 0000000..9181652 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md @@ -0,0 +1,39 @@ +## ADDED Requirements + +### Requirement: The scaffolded Vale config carries no section + +The `.vale.ini` written when a project is scaffolded SHALL contain `StylesPath` and `MinAlertLevel` and no section. A project therefore lints nothing with Vale until someone scopes something deliberately. + +An unscoped `[*]` applies every enabled rule to every file the walk reaches, which makes the default the most aggressive scope available rather than the narrowest. Scope is the author's decision, and the scaffold SHALL NOT make it on their behalf. + +#### Scenario: A freshly scaffolded project reports nothing + +- **WHEN** `check` runs against a scaffolded project with a rule file present and no section added +- **THEN** Vale SHALL report no findings +- **AND** the run SHALL NOT be reported as an engine failure + +#### Scenario: Scope is added by the author + +- **WHEN** an author scopes a rule by adding a section +- **THEN** only files matching that section SHALL be subject to it + +### Requirement: Vale diagnostics on a successful run are surfaced as notices + +When Vale exits zero and writes to stderr, the CLI SHALL surface that output as a notice on the check result. A notice SHALL NOT affect the exit code. + +This is a precondition of the section-less scaffold rather than an independent improvement. With no section to copy, the likely first edit is a rule assignment at the top level of the file, which Vale reports as ignoring — on stderr, with a zero exit and a well-formed empty result. Discarding that output leaves the author with a rule that verifies, runs, and reports nothing, which is the silent-disable failure this engine's design exists to prevent. + +#### Scenario: An ignored rule assignment reaches the user + +- **WHEN** `.vale.ini` enables a rule outside any section and `check` runs +- **THEN** the CLI SHALL surface Vale's diagnostic that the assignment was ignored + +#### Scenario: A diagnostic does not fail the check + +- **WHEN** Vale exits zero, writes a diagnostic to stderr, and reports no findings +- **THEN** the check SHALL exit zero + +#### Scenario: Silence stays silent + +- **WHEN** Vale exits zero and writes nothing to stderr +- **THEN** the CLI SHALL add no notice diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md new file mode 100644 index 0000000..eac50fc --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -0,0 +1,45 @@ +# Tasks + +## 1. Rename the command + +- [ ] 1.1 Rename `packages/cli/src/commands/help.ts` to `agent.ts` and the exported command to `agent`. Register it in `src/index.ts` +- [ ] 1.2 Remove the positional-join resolution (`positionals.join("-")`). Accept at most one positional; more than one is an error rather than a joined key. The unknown-topic message points at `taskless agent` +- [ ] 1.3 Keep the telemetry event name `cli_help` or rename it deliberately — decide once and record it, since dashboards key on it. If renamed, note it in the changeset alongside the `TOPICS` break +- [ ] 1.4 Update `help-extensions.test.ts`, `help-routing-telemetry.test.ts`, `anonymous-flag.test.ts`, `onboard.test.ts`, and `cli.test.ts` to invoke `agent` + +## 2. Rename and add the authoring topics + +- [ ] 2.1 `git mv` `help/static.txt` → `help/create-sg-rule.txt`; retitle its header and rewrite its Goal to name the artifact rather than the tier +- [ ] 2.2 `git mv` `help/existing.txt` → `help/create-legacy-rule.txt`; retitle and update its header +- [ ] 2.3 Flatten the `rule-*` topics to verb-noun single tokens (`rule-create` → `create-rule`, `rule-improve` → `improve-rule`, `rule-delete` → `delete-rule`, `rule-verify` → `verify-rule`), including their `.anonymous` variants. Decide `rule-meta` and `rule` deliberately — they are not creation verbs and may keep their names +- [ ] 2.4 Author `help/create-vale-rule.txt`: the three artifacts (style file, `.vale.ini` section, `pass/`/`fail` fixtures), that the scaffold ships section-less so the first rule writes the first scope, and that a rule enabled outside a section is ignored by Vale. State the evidence that makes `vale` the right engine for a rule, since no chooser topic states it any more. Cross-reference `verify-rule` +- [ ] 2.5 Author `help/create-runtime-rule.txt`: what a runtime rule is, where its `check.ts` lives, and why executing code requires login, reconciliation, and signing when the static tiers do not +- [ ] 2.6 Rewrite `help/route.txt` to classify into the four `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Service generation is an escalation when local authoring cannot express the rule, not a fifth destination. It must name a command the agent can run verbatim +- [ ] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`; the per-engine criterion is restated in each `create-*-rule` recipe so a consumer reading one destination knows when it applies +- [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic + +## 3. Sweep the cross-references + +- [ ] 3.1 Replace every `taskless help ` occurrence with `taskless agent ` across `src/help/*.txt`, `src/**/*.ts`, `skills/taskless/SKILL.md`, `README.md`, and `packages/cli/README.md` (~306 occurrences, 77 files). Leave `CHANGELOG.md` alone — it is a historical record +- [ ] 3.2 Update every reference to a renamed topic (`static`, `existing`, `rule create`, …) to its new single-token name +- [ ] 3.3 Add a test asserting no shipped recipe contains the string `taskless help`, and that every topic named in a recipe's See Also resolves to an embedded file. A stale cross-reference is otherwise invisible until an agent runs it + +## 4. Update the export surface + +- [ ] 4.1 `TOPICS` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]` and no longer exports `engine-selection`; move the renamed authoring topics through `INTERNAL_TOPICS` as their membership requires, keeping the two lists disjoint and jointly exhaustive over the recipe files +- [ ] 4.2 Update `prompts.test.ts` — the membership test compares against the files on disk, so it fails until the rename is complete in both places +- [ ] 4.3 Write the changeset as **MINOR** — pre-1.0, backwards-incompatible is MINOR — naming the removed topic names explicitly and stating that `@taskless/cli/prompts` consumers break on upgrade rather than at build time + +## 5. Scaffold and diagnostics (ships together) + +- [ ] 5.1 `VALE_CONFIG_CONTENT` in `0004-vale-engine.ts` drops its `[*]` section, leaving `StylesPath` and `MinAlertLevel` +- [ ] 5.2 `runVale` captures stderr on a zero-exit run and returns it as a notice on the `ok` outcome; `runValeEngine` forwards it to `DispatchResult.notices`. A notice must not touch the exit code +- [ ] 5.3 Test that a rule enabled outside a section produces a notice containing Vale's `W101` text and exits zero — this is the pairing that keeps 5.1 from reintroducing a silent disable +- [ ] 5.4 Extend the mixed-engine integration test: a scaffolded project with a rule file and no section reports nothing and does not fail; adding a section makes the same rule fire + +## 6. Verify + +- [ ] 6.1 `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli build`, `pnpm --filter @taskless/cli test` +- [ ] 6.2 Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero +- [ ] 6.3 `pnpm openspec validate --all --strict` (note: `cli-rules` and `cli-update-engine` fail on `main` already and are unrelated) +- [ ] 6.4 Archive the change From 6f1f5880470df148671b10c4b1485607b3184f0a Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Wed, 12 Aug 2026 22:20:04 -0700 Subject: [PATCH 02/19] docs(openspec): route on login state, and merge remote into create-remote-rule Settles the two open questions. `route` reads login state early, because it determines which destinations exist and discovering it late means classifying against the wrong set. It does not follow that the agent should open by asking "do you want remote generation?" -- at that point neither it nor the user knows whether the rule is a two-line pattern or something local authoring cannot express, so the question costs a turn and cannot be answered well. Service generation is offered where it is genuinely a choice: locally expressible AND logged in. Not logged in, or not locally expressible, are not choices and are not posed as one. That narrows the biased-local requirement rather than reversing it. The bias survives for the case it was written about -- local authoring that works is not abandoned for the service -- while a logged-in user stops being steered away from a path they already pay for. No recipe delegates to another. A logged-in runtime request routes straight to `create-remote-rule`; `create-runtime-rule` becomes the logged-out path and owns the gate explanation. Routing runtime through a topic that forwards would reintroduce the second fetch D1 removes, and split one explanation across two files. `remote.txt` and `rule-create.txt` merge into `create-remote-rule` -- a content merge, not a rename, since split across a boundary statement and a procedure an agent fetches one only to learn it needs the other. Adds a verification step worth more than prose review: rehearse each recipe against a subagent with no prior context, ask what steps it would take, and check that account against what the recipe intends. A recipe that reads correctly to its author and produces the wrong plan is exactly what reading it again cannot catch. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../design.md | 37 +++++++++++++++---- .../proposal.md | 4 +- .../specs/cli-agent-authoring/spec.md | 24 +++++++++++- .../specs/cli-rule-routing/spec.md | 24 ++++++++++-- .../agent-command-and-vale-authoring/tasks.md | 8 ++-- 5 files changed, 80 insertions(+), 17 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/design.md b/openspec/changes/agent-command-and-vale-authoring/design.md index 1b06336..9ccb615 100644 --- a/openspec/changes/agent-command-and-vale-authoring/design.md +++ b/openspec/changes/agent-command-and-vale-authoring/design.md @@ -18,7 +18,7 @@ Separately, the scaffolded `.vale.ini` opens an unscoped `[*]`. A whole-project **Non-Goals:** - **A `.vale.ini` writer.** The agent authors the section, exactly as it authors `sgconfig.yml` rule entries today. Construction moves to the downstream generator, consistent with `add-vale-rule-engine/design.md:107`. -- Rule _generation_ for Vale via the service. `remote` remains the service path; this change gives the local path a destination. +- Rule _generation_ for Vale via the service. `create-remote-rule` dispatches to the service as it does today; this change gives the local paths destinations and renames the remote one. - Renaming the `cli-help` capability file. The command renames; the spec keeps its name, so this change does not also move spec files (see D5). - Restricting Vale's feature set, or deciding build-output exclusion (tracked separately in #101). @@ -26,7 +26,7 @@ Separately, the scaffolded `.vale.ini` opens an unscoped `[*]`. A whole-project ### D1 — `route` and `engine-selection` merge into one topic -There is one decision, made once. `route` absorbs the engine reasoning and returns one of the four `create-*-rule` topics. `engine-selection` ceases to exist as a separate topic. +There is one decision, made once. `route` absorbs the engine reasoning and returns one of the five `create-*-rule` topics. `engine-selection` ceases to exist as a separate topic. This reverses the scoping `engine-selection` asserts today — that route decides destination, the topic decides engine, and locally the two compose. The separation is clean on paper and expensive in practice: it costs an agent two fetches and a correct handoff between them to answer one question, and the handoff is where an agent drops context. Worse, the two decisions are not independent in the direction the split assumes; "author this locally" and "which engine can express it" are answered from the same evidence, so splitting them means reading the same signals twice. @@ -40,7 +40,7 @@ _Alternative rejected:_ export `route`. It is local-only by construction. ### D2 — Verb-noun names, single token, no aliases -`create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, `create-legacy-rule`. Not everything needs the noun — `route` stays `route`. +`create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, `create-legacy-rule`, `create-remote-rule`. Not everything needs the noun — `route` stays `route`. Hyphenated single tokens are the point rather than a side effect. A multi-word phrase invites an agent to paraphrase or reorder; a hyphenated token reads as a literal string to copy. This is the same reason the resolution stops joining positionals: with one token there is no order to get wrong. @@ -86,18 +86,39 @@ No user data or on-disk state migrates. Existing projects keep whatever `.vale.i The rename is a hard cutover in one PR: recipes cross-reference each other by literal command, so a partial rename produces recipes pointing at commands that do not exist. -### D6 — There is no `create-remote-rule`; `create-runtime-rule` covers the gated path +### D6 — The logged-out gate is explained once, by `create-runtime-rule` -`route` names four destinations, all engine-shaped. "Remote" is not a fifth engine — it describes who generates, which is a different question, and the one place it bites is the same place `create-runtime-rule` already has to speak: the user is logged out. +"Remote" describes who generates rather than which engine, so the two are not peers in kind — but the place that matters is the same place `create-runtime-rule` has to speak anyway: the user is logged out. -So `create-runtime-rule` owns the gated story end to end — why executing code requires login, reconciliation, and signing, and what to do when the user has none of them. A logged-out user meets one topic explaining one gate, rather than being routed to a topic about remoteness that then explains authentication. +`create-runtime-rule` therefore owns the gated story — why executing code requires login, reconciliation, and signing, and how to get there. A logged-out user meets one topic explaining one gate, rather than being handed between a topic about remoteness and one about authentication. **What this leaves unresolved, deliberately.** `remote.txt` and `rule-create.txt` today serve a real and different flow: the service generating an _ast-grep_ rule when local authoring cannot. That is escalation, not a destination — `route` is already specified as biased local with the service as last resort — so it survives as a fallback inside `route` rather than as a peer of the four. Whether those two recipes keep their names, merge, or fold into `create-sg-rule`'s failure path is not settled here. _Alternative rejected:_ a fifth `create-remote-rule` destination. Puts a non-engine on an engine-shaped list, and splits the logged-out explanation across two topics. +### D7 — Login state is read early; remote is offered only where it is a choice + +`route` reads login state near the top, before dispatching. It changes which destinations exist, so discovering it late means classifying against a set that may be wrong. + +It does **not** follow that the agent should open by asking "do you want remote generation?". At that point it does not know whether the rule is a two-line `sg` pattern or something local authoring cannot express, and neither does the user — the question costs a turn and cannot be answered well. Remote is offered when it is genuinely a choice: the rule is locally expressible **and** the user is logged in. Not logged in, or not locally expressible, are not choices and are not posed as one. + +This narrows the existing "biased to stay local" requirement rather than reversing it. The bias survives for the case it was written about — local authoring that _works_ is not abandoned for the service — while a logged-in user stops being steered away from a path they have already paid for. + +**No topic delegates to another.** A logged-in runtime request routes to `create-remote-rule` directly; `create-runtime-rule` is the logged-out path. Routing runtime through a topic that then forwards would reintroduce the second fetch D1 exists to remove, and would split the login explanation across two files. + +_Alternative rejected:_ offer remote unconditionally as step one. Reverses the local bias outright, and asks a question before the information exists to answer it. + +_Alternative rejected:_ `create-runtime-rule` checks login and forwards. Two fetches, and the reader meets the gate explanation only on one branch. + +### D8 — `remote` and `rule-create` merge into `create-remote-rule` + +`remote.txt` states the client-side boundary; `rule-create.txt` is the procedure that enriches a description, calls the API, and reports. Split across two topics, an agent fetches one to learn it needs the other. + +This is a content merge, not a rename: both texts have material that survives, and the result has to read as one procedure rather than two concatenated. + +_Alternative rejected:_ keep `remote` as a boundary statement and `create-remote-rule` as the procedure. Preserves the second fetch under new names. + ## Open Questions -- `remote.txt` and `rule-create.txt` are no longer named by `route`, but still implement service generation. Do they remain fetchable topics, fold into `create-sg-rule`'s escalation path, or merge with each other? Not settled by D6, and worth deciding before the sweep in task group 3 rewrites their cross-references. -- Should `create-runtime-rule` explain the login requirement itself, or defer to `auth`? Stated as the former; worth confirming it does not duplicate `auth`. +- Does `create-runtime-rule` duplicate `auth`? It now owns the login explanation for the runtime tier specifically; worth confirming the two do not drift. - Does the distributed engine criterion (D1) belong in each recipe's Preconditions or as a named section? Affects whether a consumer can extract it mechanically. diff --git a/openspec/changes/agent-command-and-vale-authoring/proposal.md b/openspec/changes/agent-command-and-vale-authoring/proposal.md index b6e209e..f64115b 100644 --- a/openspec/changes/agent-command-and-vale-authoring/proposal.md +++ b/openspec/changes/agent-command-and-vale-authoring/proposal.md @@ -13,9 +13,11 @@ The same work exposed that the surface an agent reads is shaped for a human. `ta - `existing` → `create-legacy-rule` - new `create-vale-rule` - new `create-runtime-rule` + - `remote` + `rule-create` merge into `create-remote-rule` - **BREAKING** — `TOPICS` in `@taskless/cli/prompts` renames with them. `["static", "engine-selection"]` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]`. No alias is kept. Pre-1.0, a backwards-incompatible change is a **MINOR** bump. -- `route` becomes the single front door, returning a concrete next command rather than a category. Its decision set is the four `create-*-rule` topics; service generation stays an escalation for when local authoring cannot express the rule, not a peer destination. +- `route` becomes the single front door, returning a concrete next command rather than a category. Its decision set is the five `create-*-rule` topics. It reads login state early, because that changes which destinations exist, and offers service generation **only where it is a real choice** — when the rule is locally expressible and the user is logged in. A user who is not logged in, or a rule local authoring cannot express, is not being offered anything. - **BREAKING** — `engine-selection` merges into `route` and stops existing as a topic. Its criterion distributes: `route` applies it to dispatch, and each `create-*-rule` recipe states the evidence that makes its own engine right. That is what keeps it exportable — a consumer outside the CLI has no `route` step and cannot run `taskless detect --json`, so a chooser topic was unusable to it anyway. +- `create-runtime-rule` becomes the logged-**out** path: what a runtime rule is, why executing code requires login, reconciliation, and signing, and how to get there. A logged-in runtime request goes straight to `create-remote-rule` from `route`, so no topic delegates to another. - `create-vale-rule` covers what no topic covers today: authoring a Vale style file under `vale/rules/`, scoping it with a `.vale.ini` section, and writing `pass/`/`fail` fixtures. Consistent with `create-sg-rule`, the agent writes these files; no CLI writer is introduced. - The scaffolded `.vale.ini` ships **no section**, so a fresh project lints nothing until a user scopes something deliberately. `create-vale-rule` teaches writing that first section. - Vale's stderr diagnostics on a successful run surface as notices. This is required by the change above, not incidental: with no section to copy, the likely first mistake is a rule assignment at top level, which Vale reports as `W101 ... is ignoring it` on stderr and which today is discarded — reproducing the silent-disable class the Vale work exists to eliminate. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md index 1aad0d1..7a2f382 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md @@ -53,11 +53,31 @@ This matches how ast-grep rules are authored today: the agent writes the rule an - **THEN** it edits `.vale.ini` itself - **AND** the CLI SHALL NOT offer a subcommand that performs that edit -### Requirement: The runtime authoring recipe states why it is gated +### Requirement: The runtime authoring recipe is the logged-out path -The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, and SHALL state this as a property of executing code rather than as a property of the engine's capability. +The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, SHALL state this as a property of executing code rather than of the engine's capability, and SHALL say how to obtain access. + +It SHALL NOT forward the agent to another authoring recipe. A logged-in runtime request is routed to `create-remote-rule` by `route`, so this recipe is reached only when the gate is closed and exists to explain that one gate once. #### Scenario: The gate is explained where it is encountered - **WHEN** an agent follows `create-runtime-rule` - **THEN** the recipe SHALL state why the runtime tier is gated when the static tiers are not +- **AND** it SHALL state how to obtain access + +#### Scenario: The recipe does not delegate + +- **WHEN** an agent follows `create-runtime-rule` +- **THEN** it SHALL NOT be directed to fetch another authoring recipe to proceed + +### Requirement: Service generation is one recipe + +The CLI SHALL provide a single `create-remote-rule` recipe covering both the client-side boundary of service generation and the procedure itself — enriching the user's description, dispatching to the Taskless service, and reporting the result. + +Split across a boundary statement and a procedure, an agent fetches one only to learn it needs the other, which is the second fetch this change exists to remove. + +#### Scenario: One fetch reaches the whole procedure + +- **WHEN** an agent follows `create-remote-rule` +- **THEN** the recipe SHALL carry both the boundary and the dispatch procedure +- **AND** it SHALL NOT require fetching a second topic to complete the request diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md index 0b18fd9..62cf0ce 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md @@ -2,7 +2,7 @@ ### Requirement: Route is the local authoring classifier -The CLI SHALL provide a `route` help recipe that instructs the agent to classify a rule-authoring request into one of four destinations — `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, or `create-runtime-rule` — using `taskless detect --json` signals plus the user's intent. The `route` recipe SHALL be biased to stay local, treating service generation as an escalation available when local authoring cannot express the rule rather than as a peer destination. +The CLI SHALL provide a `route` help recipe that instructs the agent to classify a rule-authoring request into one of five destinations — `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, or `create-remote-rule` — using `taskless detect --json` signals plus the user's intent. The `route` recipe SHALL read the user's login state before dispatching, since it determines which destinations are reachable. It SHALL remain biased to stay local: local authoring that works SHALL NOT be abandoned for the service. `route` SHALL decide the engine as part of this classification rather than deferring it to a separate topic. There is one decision, made from one reading of the evidence: whether a rule is expressible locally and which engine can express it are answered from the same signals, so splitting them costs a second fetch and a handoff without adding information. @@ -14,10 +14,10 @@ Each destination SHALL be a topic an agent can fetch by name, so classifying pro - **THEN** the recipe SHALL direct the agent to run `taskless detect --json` and use its signals as input to the classification -#### Scenario: Route classifies into one of four destinations +#### Scenario: Route classifies into one of five destinations - **WHEN** the agent follows `route` -- **THEN** it SHALL select exactly one of `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, or `create-runtime-rule` +- **THEN** it SHALL select exactly one of `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, or `create-remote-rule` - **AND** it SHALL fetch the corresponding recipe to perform the authoring #### Scenario: Every destination resolves to a recipe @@ -25,6 +25,24 @@ Each destination SHALL be a topic an agent can fetch by name, so classifying pro - **WHEN** any destination `route` can name is fetched - **THEN** a recipe of that exact name SHALL exist +#### Scenario: Service generation is offered only where it is a choice + +- **WHEN** the rule is expressible locally AND the user is logged in +- **THEN** `route` MAY offer `create-remote-rule` as an alternative and ask the user +- **AND WHEN** the user is not logged in, or the rule is not expressible locally +- **THEN** `route` SHALL NOT pose service generation as a choice, because it is not one + +#### Scenario: A logged-in runtime request routes straight to the service + +- **WHEN** the rule requires the runtime engine AND the user is logged in +- **THEN** `route` SHALL name `create-remote-rule` +- **AND** no recipe SHALL forward the agent from one destination to another + +#### Scenario: A logged-out runtime request reaches the explanation + +- **WHEN** the rule requires the runtime engine AND the user is not logged in +- **THEN** `route` SHALL name `create-runtime-rule` + #### Scenario: The engine is decided without a second fetch - **WHEN** the agent follows `route` diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index eac50fc..9baffeb 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -13,8 +13,9 @@ - [ ] 2.2 `git mv` `help/existing.txt` → `help/create-legacy-rule.txt`; retitle and update its header - [ ] 2.3 Flatten the `rule-*` topics to verb-noun single tokens (`rule-create` → `create-rule`, `rule-improve` → `improve-rule`, `rule-delete` → `delete-rule`, `rule-verify` → `verify-rule`), including their `.anonymous` variants. Decide `rule-meta` and `rule` deliberately — they are not creation verbs and may keep their names - [ ] 2.4 Author `help/create-vale-rule.txt`: the three artifacts (style file, `.vale.ini` section, `pass/`/`fail` fixtures), that the scaffold ships section-less so the first rule writes the first scope, and that a rule enabled outside a section is ignored by Vale. State the evidence that makes `vale` the right engine for a rule, since no chooser topic states it any more. Cross-reference `verify-rule` -- [ ] 2.5 Author `help/create-runtime-rule.txt`: what a runtime rule is, where its `check.ts` lives, and why executing code requires login, reconciliation, and signing when the static tiers do not -- [ ] 2.6 Rewrite `help/route.txt` to classify into the four `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Service generation is an escalation when local authoring cannot express the rule, not a fifth destination. It must name a command the agent can run verbatim +- [ ] 2.5 Author `help/create-runtime-rule.txt` as the logged-**out** path: what a runtime rule is, where its `check.ts` lives, why executing code requires login, reconciliation, and signing when the static tiers do not, and how to obtain access. It must not forward to another recipe +- [ ] 2.5a Merge `help/remote.txt` and `help/rule-create.txt` into `help/create-remote-rule.txt` (plus the `.anonymous` variant). A content merge, not a rename: both texts have material that survives, and the result must read as one procedure rather than two concatenated +- [ ] 2.6 Rewrite `help/route.txt` to read login state early and classify into the five `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Offer `create-remote-rule` only where it is a genuine choice — locally expressible AND logged in. A logged-in runtime request routes straight to `create-remote-rule`; a logged-out one to `create-runtime-rule`. It must name a command the agent can run verbatim - [ ] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`; the per-engine criterion is restated in each `create-*-rule` recipe so a consumer reading one destination knows when it applies - [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic @@ -40,6 +41,7 @@ ## 6. Verify - [ ] 6.1 `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli build`, `pnpm --filter @taskless/cli test` -- [ ] 6.2 Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero +- [ ] 6.2 **Rehearse the recipes against a fresh agent.** For `route` and each `create-*-rule`, hand the text to a subagent with no prior context and ask it to describe the steps it would take and the commands it would run. Check its account against what the recipe intends — a recipe that reads correctly to its author and produces the wrong plan is the failure mode prose review cannot catch. Feed the corrections back into the text +- [ ] 6.2a Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero - [ ] 6.3 `pnpm openspec validate --all --strict` (note: `cli-rules` and `cli-update-engine` fail on `main` already and are unrelated) - [ ] 6.4 Archive the change From d467e42b04cae4fc564a1f7f851c5ce08f1bbb50 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Wed, 12 Aug 2026 22:26:43 -0700 Subject: [PATCH 03/19] docs(openspec): state the engine criterion once, and defer login to auth Closes both open questions, and corrects an overclaim in D1 while doing it. D1 justified merging the chooser partly on the criterion distributing to the destinations, each stating the evidence for its own engine. That is the same duplication the merge exists to remove, one level down: five copies of one test, and the first edit to any of them diverges silently. The criterion now lives once, in `route`'s destination table, which is where the comparison is actually made. Destinations orient instead. Each opens with a fixed line -- what topic this is, what it helps you write, revisit routing if that is not what you need. Its job is self-correction: an agent that arrived wrongly, by guessing or by a user naming a topic directly, should find out in the first line where recovery is cheap, not after authoring the wrong artifact. Scope only, never the comparison. The cost is worth stating rather than hiding. A consumer reading only `create-vale-rule` gets its scope but not the boundary cases that settle hard calls. Enough to pick between destinations; not enough to adjudicate an ambiguous rule. If the generator needs the full test it consumes `route` or keeps its own classifier -- recorded as the one consumer this change leaves with less than it had, rather than papered over by copying prose into five files. `create-runtime-rule` points at `auth` rather than restating login. An extra CLI turn is cheap when each turn delivers something concrete, and a copy of the login procedure inside a rule-authoring recipe goes stale the first time login changes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../design.md | 25 ++++++++++++++++--- .../specs/cli-agent-authoring/spec.md | 21 ++++++++++++++-- .../specs/cli-rule-routing/spec.md | 10 ++++---- .../agent-command-and-vale-authoring/tasks.md | 5 ++-- 4 files changed, 49 insertions(+), 12 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/design.md b/openspec/changes/agent-command-and-vale-authoring/design.md index 9ccb615..ef2152e 100644 --- a/openspec/changes/agent-command-and-vale-authoring/design.md +++ b/openspec/changes/agent-command-and-vale-authoring/design.md @@ -32,10 +32,16 @@ This reverses the scoping `engine-selection` asserts today — that route decide **Where the reasoning goes for consumers outside the CLI.** The platform generator consumes `engine-selection` through `TOPICS` and has no `route` step, so merging cannot simply delete what it reads. Exporting `route` instead is wrong — it is built on `taskless detect --json` and local authoring, neither of which a Worker can do. -The reasoning therefore **distributes to the destinations**: each `create-*-rule` recipe states the evidence that makes its engine the right one, and `route` applies those same tests to dispatch. A consumer choosing between `create-sg-rule` and `create-vale-rule` reads the criterion at the destination rather than fetching a chooser first. This keeps the exported surface honest — every exported topic is one a service-side consumer can actually act on — and it removes the class of drift where a chooser and its destinations disagree about when each applies. +The criterion therefore lives **once, in `route`'s destination table**, which is where the comparison is actually made. Restating it in each destination would be the drift risk the merge was meant to remove, one level down. + +Each destination instead opens with a short orientation line naming what it is for and what to do if that is wrong — see D9. That is deliberately less than the full criterion: enough for a reader who arrived at the wrong recipe to notice and go back, not a second copy of the test. + +**What this costs the exported surface, stated plainly.** A consumer reading only `create-vale-rule` gets its scope ("prose and markup") but not the boundary cases that distinguish hard calls — prose-about-code, per-document versus cross-document. For picking between two destinations that is sufficient; for adjudicating a genuinely ambiguous rule it is not. If the generator needs the full test it will have to consume `route` or keep its own classifier, and that is a decision for the generator rather than something this change should pre-empt by duplicating prose. _Alternative rejected:_ keep `engine-selection` as a third exported topic that `route` also applies. Two statements of the same criterion, guaranteed to drift, and it preserves the second fetch for exactly the consumer we were trying to simplify. +_Alternative rejected:_ restate the full criterion in every destination. Five copies of one test, and the first edit to any of them is a divergence nobody notices. + _Alternative rejected:_ export `route`. It is local-only by construction. ### D2 — Verb-noun names, single token, no aliases @@ -118,7 +124,20 @@ This is a content merge, not a rename: both texts have material that survives, a _Alternative rejected:_ keep `remote` as a boundary statement and `create-remote-rule` as the procedure. Preserves the second fetch under new names. +### D9 — Destinations orient, they do not re-decide + +Each `create-*-rule` recipe opens with a fixed-shape line: what topic the reader is in, what kinds of rule it helps write, and an instruction to revisit the routing decision if that is not what they need. + +Its job is self-correction, not classification. An agent that arrived at the wrong recipe — because it guessed, because a user named a topic directly, or because `route` was wrong — should discover that in the first line rather than after authoring the wrong artifact. Recovery is cheap there and expensive later. + +Keeping it to orientation is what stops it becoming a second criterion. The comparison between engines happens in one place; a destination only has to answer "am I the right place", which needs its own scope and nothing about the others. + +### D10 — `create-runtime-rule` defers to `auth` + +It explains why the runtime tier is gated — executing code requires reconciliation and signing — and points at `auth` for obtaining access, rather than restating the login procedure. + +An extra CLI turn is not a cost worth avoiding when each turn delivers something concrete: `auth` is maintained as the authority on login, and a copy inside a rule-authoring recipe is a copy that goes stale the first time login changes. + ## Open Questions -- Does `create-runtime-rule` duplicate `auth`? It now owns the login explanation for the runtime tier specifically; worth confirming the two do not drift. -- Does the distributed engine criterion (D1) belong in each recipe's Preconditions or as a named section? Affects whether a consumer can extract it mechanically. +- If the platform generator needs the full engine criterion rather than per-destination scope, does it consume `route` or keep its own classifier? Out of scope here (D1), but it is the one consumer this change leaves with less than it had. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md index 7a2f382..1b6340f 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md @@ -55,7 +55,7 @@ This matches how ast-grep rules are authored today: the agent writes the rule an ### Requirement: The runtime authoring recipe is the logged-out path -The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, SHALL state this as a property of executing code rather than of the engine's capability, and SHALL say how to obtain access. +The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, and SHALL state this as a property of executing code rather than of the engine's capability. It SHALL point at `auth` for obtaining access rather than restating the login procedure, which `auth` owns. It SHALL NOT forward the agent to another authoring recipe. A logged-in runtime request is routed to `create-remote-rule` by `route`, so this recipe is reached only when the gate is closed and exists to explain that one gate once. @@ -63,7 +63,7 @@ It SHALL NOT forward the agent to another authoring recipe. A logged-in runtime - **WHEN** an agent follows `create-runtime-rule` - **THEN** the recipe SHALL state why the runtime tier is gated when the static tiers are not -- **AND** it SHALL state how to obtain access +- **AND** it SHALL refer the reader to `auth` rather than restating how to log in #### Scenario: The recipe does not delegate @@ -81,3 +81,20 @@ Split across a boundary statement and a procedure, an agent fetches one only to - **WHEN** an agent follows `create-remote-rule` - **THEN** the recipe SHALL carry both the boundary and the dispatch procedure - **AND** it SHALL NOT require fetching a second topic to complete the request + +### Requirement: Every authoring recipe opens by orienting the reader + +Each `create-*-rule` recipe SHALL open with a line naming the topic the reader is in, the kinds of rule it helps write, and an instruction to revisit the routing decision if that is not what they need. + +The line SHALL orient, not classify: it states this recipe's own scope and SHALL NOT restate the criterion distinguishing the engines from each other, which `route` holds in one place. An agent that arrived at the wrong recipe — by guessing, by a user naming a topic directly, or because `route` was wrong — should discover it in the first line, where recovery is cheap, rather than after authoring the wrong artifact. + +#### Scenario: A misrouted reader is told how to recover + +- **WHEN** an agent opens any `create-*-rule` recipe +- **THEN** the first lines SHALL name what that recipe helps write +- **AND** SHALL instruct the agent to revisit its routing decision if it needs a different kind of check + +#### Scenario: The orientation is not a second criterion + +- **WHEN** the orientation line is read +- **THEN** it SHALL describe only this recipe's scope, not the comparison between engines diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md index 62cf0ce..85c90f6 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md @@ -77,14 +77,14 @@ Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to ### Requirement: Engine reasoning lives in route and in each destination -The engine criterion SHALL be stated in `route`, which applies it to dispatch, and in each `create-*-rule` recipe, which states the evidence that makes its own engine the right one. It SHALL NOT be stated in a separate chooser topic. +The engine criterion SHALL be stated once, in `route`'s destination table, which is where the comparison between engines is made. It SHALL NOT be stated in a separate chooser topic, and SHALL NOT be restated in the destination recipes. -Distributing it this way is what lets the reasoning stay exportable. A consumer outside the CLI has no `route` step and cannot run `taskless detect --json`, so a chooser topic is unusable to it; a destination that states its own criterion is usable by anything that can read one recipe. It also removes the drift class where a chooser and its destinations disagree about when each applies. +One statement is the point. A criterion copied into each destination is five copies of one test, and the first edit to any of them is a divergence nobody notices — the drift this merge exists to remove, reappearing one level down. Destinations orient the reader to their own scope instead, which needs nothing about the other engines. -#### Scenario: A destination states when it applies +#### Scenario: The comparison lives in one place -- **WHEN** an agent or consumer reads `create-sg-rule` or `create-vale-rule` -- **THEN** the recipe SHALL state the evidence that makes that engine the right one for a rule +- **WHEN** the embedded recipe set is inspected +- **THEN** exactly one recipe SHALL state the criterion distinguishing the engines from each other #### Scenario: No separate chooser topic exists diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 9baffeb..89c87f4 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -13,10 +13,11 @@ - [ ] 2.2 `git mv` `help/existing.txt` → `help/create-legacy-rule.txt`; retitle and update its header - [ ] 2.3 Flatten the `rule-*` topics to verb-noun single tokens (`rule-create` → `create-rule`, `rule-improve` → `improve-rule`, `rule-delete` → `delete-rule`, `rule-verify` → `verify-rule`), including their `.anonymous` variants. Decide `rule-meta` and `rule` deliberately — they are not creation verbs and may keep their names - [ ] 2.4 Author `help/create-vale-rule.txt`: the three artifacts (style file, `.vale.ini` section, `pass/`/`fail` fixtures), that the scaffold ships section-less so the first rule writes the first scope, and that a rule enabled outside a section is ignored by Vale. State the evidence that makes `vale` the right engine for a rule, since no chooser topic states it any more. Cross-reference `verify-rule` -- [ ] 2.5 Author `help/create-runtime-rule.txt` as the logged-**out** path: what a runtime rule is, where its `check.ts` lives, why executing code requires login, reconciliation, and signing when the static tiers do not, and how to obtain access. It must not forward to another recipe +- [ ] 2.5 Author `help/create-runtime-rule.txt` as the logged-**out** path: what a runtime rule is, where its `check.ts` lives, and why executing code requires login, reconciliation, and signing when the static tiers do not. Point at `auth` for obtaining access rather than restating it. It must not forward to another authoring recipe - [ ] 2.5a Merge `help/remote.txt` and `help/rule-create.txt` into `help/create-remote-rule.txt` (plus the `.anonymous` variant). A content merge, not a rename: both texts have material that survives, and the result must read as one procedure rather than two concatenated - [ ] 2.6 Rewrite `help/route.txt` to read login state early and classify into the five `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Offer `create-remote-rule` only where it is a genuine choice — locally expressible AND logged in. A logged-in runtime request routes straight to `create-remote-rule`; a logged-out one to `create-runtime-rule`. It must name a command the agent can run verbatim -- [ ] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`; the per-engine criterion is restated in each `create-*-rule` recipe so a consumer reading one destination knows when it applies +- [ ] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`'s destination table — stated once, not copied into the destinations +- [ ] 2.7a Give every `create-*-rule` recipe the same opening orientation line: what topic this is, what it helps you write, and revisit routing if that is not what you need. Fixed shape across all five so an agent recognises it; scope only, never the comparison between engines - [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic ## 3. Sweep the cross-references From d736f46c3fe0ad82a00c0dbe744ce1896eb01a92 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Wed, 12 Aug 2026 22:27:24 -0700 Subject: [PATCH 04/19] docs(openspec): record why the prompts are exported, and soften the route claim D1 asserted that exporting `route` is wrong because it is built on local mechanics. That is too strong: `route` is expected to be exported in a later change. It is not exported *here* because its local mechanics need separating from its reasoning first, and doing that inside a change that already renames a command and five topics is how a rename becomes unreviewable. Recorded as deferred rather than rejected. That also closes the last open question. The gap this change leaves -- destinations carry scope, not the boundary cases -- closes when `route` is exported: the service holds the route prompt, needs no escalation path because it is the escalation, and supplies its own runtime prompt for its own agentic flow. Worth stating why the prompts are exported at all, since it changes what "enough" means. The goal is consistency between the local and remote paths, not transferring a capability the service lacks. The service can classify without us; what it must not do is classify differently. A rule routed to `vale` locally and `sg` server-side is one request answered two ways, and that is what the shared surface exists to prevent. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../agent-command-and-vale-authoring/design.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/design.md b/openspec/changes/agent-command-and-vale-authoring/design.md index ef2152e..9e37534 100644 --- a/openspec/changes/agent-command-and-vale-authoring/design.md +++ b/openspec/changes/agent-command-and-vale-authoring/design.md @@ -30,19 +30,23 @@ There is one decision, made once. `route` absorbs the engine reasoning and retur This reverses the scoping `engine-selection` asserts today — that route decides destination, the topic decides engine, and locally the two compose. The separation is clean on paper and expensive in practice: it costs an agent two fetches and a correct handoff between them to answer one question, and the handoff is where an agent drops context. Worse, the two decisions are not independent in the direction the split assumes; "author this locally" and "which engine can express it" are answered from the same evidence, so splitting them means reading the same signals twice. -**Where the reasoning goes for consumers outside the CLI.** The platform generator consumes `engine-selection` through `TOPICS` and has no `route` step, so merging cannot simply delete what it reads. Exporting `route` instead is wrong — it is built on `taskless detect --json` and local authoring, neither of which a Worker can do. +**Where the reasoning goes for consumers outside the CLI.** The platform generator consumes `engine-selection` through `TOPICS`, so merging cannot simply delete what it reads. `route` is expected to be exported in a later change, at which point it carries the criterion to the service directly; it is not exported here because it still contains local mechanics (`taskless detect --json`, on-device authoring) that a Worker cannot run, and untangling those is its own piece of work. The criterion therefore lives **once, in `route`'s destination table**, which is where the comparison is actually made. Restating it in each destination would be the drift risk the merge was meant to remove, one level down. Each destination instead opens with a short orientation line naming what it is for and what to do if that is wrong — see D9. That is deliberately less than the full criterion: enough for a reader who arrived at the wrong recipe to notice and go back, not a second copy of the test. -**What this costs the exported surface, stated plainly.** A consumer reading only `create-vale-rule` gets its scope ("prose and markup") but not the boundary cases that distinguish hard calls — prose-about-code, per-document versus cross-document. For picking between two destinations that is sufficient; for adjudicating a genuinely ambiguous rule it is not. If the generator needs the full test it will have to consume `route` or keep its own classifier, and that is a decision for the generator rather than something this change should pre-empt by duplicating prose. +**What this costs the exported surface, and for how long.** A consumer reading only `create-vale-rule` gets its scope ("prose and markup") but not the boundary cases that settle hard calls — prose-about-code, per-document versus cross-document. Sufficient for picking between destinations; not for adjudicating a genuinely ambiguous rule. + +That gap closes when `route` is exported. The service will hold the route prompt, which states when each engine applies, and needs no escalation path of its own — it is the escalation. It can then supply its own runtime prompt for its own agentic flow. + +Worth being clear about why the prompts are exported at all, because it changes what "enough" means: the goal is **consistency between the local and remote paths**, not transferring a capability the service lacks. The service can classify without us. What it should not do is classify _differently_ — a rule routed to `vale` locally and to `sg` server-side is the same request answered two ways, and that is the failure the shared surface exists to prevent. _Alternative rejected:_ keep `engine-selection` as a third exported topic that `route` also applies. Two statements of the same criterion, guaranteed to drift, and it preserves the second fetch for exactly the consumer we were trying to simplify. _Alternative rejected:_ restate the full criterion in every destination. Five copies of one test, and the first edit to any of them is a divergence nobody notices. -_Alternative rejected:_ export `route`. It is local-only by construction. +_Deferred, not rejected:_ export `route`. Its local mechanics need separating from its reasoning first, and doing that inside a change that already renames a command and five topics is how a rename becomes unreviewable. ### D2 — Verb-noun names, single token, no aliases @@ -140,4 +144,4 @@ An extra CLI turn is not a cost worth avoiding when each turn delivers something ## Open Questions -- If the platform generator needs the full engine criterion rather than per-destination scope, does it consume `route` or keep its own classifier? Out of scope here (D1), but it is the one consumer this change leaves with less than it had. +- None outstanding. From 093aca1583d5a9f8b1e45325e015160ca29a66f1 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Wed, 12 Aug 2026 23:34:17 -0700 Subject: [PATCH 05/19] ref(cli): rename the `help` command to `agent`, one token per topic MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The subcommand is named for its reader: it serves agents fetching a procedure, not humans asking for help. Topics are now addressed by exactly one token. Joining positionals made `taskless help rule create` resolve `rule-create.txt`, which invited an agent to reorder or paraphrase a topic name and still get a hit. A single hyphenated token is a literal string to copy, so extra positionals are an error rather than something to guess at. The `cli_help` telemetry event name is deliberately unchanged — dashboards key on it, and renaming it alongside the upcoming `TOPICS` break would take them dark for an unrelated reason. Recipe text still says `taskless help`; that sweep is a later task group. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../agent-command-and-vale-authoring/tasks.md | 10 ++- .../cli/src/commands/{help.ts => agent.ts} | 51 +++++++---- packages/cli/src/commands/rules.ts | 4 +- packages/cli/src/index.ts | 14 +-- packages/cli/src/telemetry-run.ts | 4 +- packages/cli/test/anonymous-flag.test.ts | 4 +- packages/cli/test/cli-run.test.ts | 2 +- packages/cli/test/cli.test.ts | 2 +- packages/cli/test/help-extensions.test.ts | 87 +++++++++++-------- .../cli/test/help-routing-telemetry.test.ts | 10 +-- packages/cli/test/help-telemetry.test.ts | 26 ++++-- packages/cli/test/onboard.test.ts | 8 +- packages/cli/test/prompts.test.ts | 6 +- 13 files changed, 136 insertions(+), 92 deletions(-) rename packages/cli/src/commands/{help.ts => agent.ts} (71%) diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 89c87f4..1b3ec3a 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -2,10 +2,12 @@ ## 1. Rename the command -- [ ] 1.1 Rename `packages/cli/src/commands/help.ts` to `agent.ts` and the exported command to `agent`. Register it in `src/index.ts` -- [ ] 1.2 Remove the positional-join resolution (`positionals.join("-")`). Accept at most one positional; more than one is an error rather than a joined key. The unknown-topic message points at `taskless agent` -- [ ] 1.3 Keep the telemetry event name `cli_help` or rename it deliberately — decide once and record it, since dashboards key on it. If renamed, note it in the changeset alongside the `TOPICS` break -- [ ] 1.4 Update `help-extensions.test.ts`, `help-routing-telemetry.test.ts`, `anonymous-flag.test.ts`, `onboard.test.ts`, and `cli.test.ts` to invoke `agent` +- [x] 1.1 Rename `packages/cli/src/commands/help.ts` to `agent.ts` and the exported command to `agent`. Register it in `src/index.ts` +- [x] 1.2 Remove the positional-join resolution (`positionals.join("-")`). Accept at most one positional; more than one is an error rather than a joined key. The unknown-topic message points at `taskless agent` +- [x] 1.3 Keep the telemetry event name `cli_help` or rename it deliberately — decide once and record it, since dashboards key on it. If renamed, note it in the changeset alongside the `TOPICS` break + - **Decision: keep `cli_help`.** Renaming it in the same change that breaks the `TOPICS` export would take the dashboards dark for a reason unrelated to this change, and agent-call volume needs to stay visible under the existing event. The event name is not part of any agent-facing contract, so it can be renamed later on its own. Recorded as a comment at the capture site in `agent.ts`; nothing to add to the changeset +- [x] 1.4 Update `help-extensions.test.ts`, `help-routing-telemetry.test.ts`, `anonymous-flag.test.ts`, `onboard.test.ts`, and `cli.test.ts` to invoke `agent` + - Also required, not listed: `help-telemetry.test.ts` (imports `createHelpCommand` directly), `prompts.test.ts` (spawns `binPath help ` for the parity test), and `cli-run.test.ts` (its `resolveCommandName` case named `help`) ## 2. Rename and add the authoring topics diff --git a/packages/cli/src/commands/help.ts b/packages/cli/src/commands/agent.ts similarity index 71% rename from packages/cli/src/commands/help.ts rename to packages/cli/src/commands/agent.ts index d89e4c9..e50647b 100644 --- a/packages/cli/src/commands/help.ts +++ b/packages/cli/src/commands/agent.ts @@ -10,8 +10,8 @@ import { import { getTelemetry } from "../telemetry"; import { getRecipe } from "../prompts/recipes"; -// Help-only recipe topics (no backing subcommand) that should still be -// discoverable from the `taskless help` index. The rule-authoring front +// Recipe-only topics (no backing subcommand) that should still be +// discoverable from the `taskless agent` index. The rule-authoring front // door (`route`) and its destinations live here so an agent can find them. const RECIPE_TOPICS: ReadonlyArray<[string, string]> = [ ["route", "Decide where to author a rule (existing/static/remote)"], @@ -36,11 +36,11 @@ async function resolveDescription( return meta?.description ?? ""; } -export function createHelpCommand(subCommands: SubCommandsDef) { +export function createAgentCommand(subCommands: SubCommandsDef) { return defineCommand({ meta: { - name: "help", - description: "Show help for a command", + name: "agent", + description: "Return a recipe for an AI coding agent to follow", }, args: { dir: { @@ -67,14 +67,19 @@ export function createHelpCommand(subCommands: SubCommandsDef) { if (!argument.includes("=") && valueFlagSet.has(argument)) index++; continue; } - if (argument !== "help") positionals.push(argument); + if (argument !== "agent") positionals.push(argument); } const cwd = resolve(args.dir); const telemetry = await getTelemetry(cwd); if (positionals.length === 0) { - // cli_help with the index marker: agent fetched the topic list + // cli_help with the index marker: agent fetched the topic list. + // The event name stays `cli_help` even though the command is now + // `agent`: dashboards key on it, it is not part of any agent-facing + // contract, and renaming it in the same change that breaks the + // `TOPICS` export would take those dashboards dark for a reason + // unrelated to this change. telemetry.capture("cli_help", { topic: "(index)" }); console.log("Taskless CLI\n"); @@ -89,7 +94,7 @@ export function createHelpCommand(subCommands: SubCommandsDef) { const entries: Array<[string, string]> = []; for (const [name, cmd] of Object.entries(subCommands)) { - if (name === "help") continue; + if (name === "agent") continue; const description = await resolveDescription(cmd); entries.push([name, description]); } @@ -114,30 +119,44 @@ export function createHelpCommand(subCommands: SubCommandsDef) { ); console.log("and use local-only behavior."); console.log( - "\nRun `taskless help ` for the full recipe (e.g. `taskless help rule create`)." + "\nRun `taskless agent ` for the full recipe (e.g. `taskless agent rule-create`)." ); return; } - // Join positional args to form the lookup key - const key = positionals.join("-"); + // Topics are addressed by exactly one token. Joining positionals into a + // key used to make `rule create` resolve `rule-create.txt`, which invited + // an agent to reorder or paraphrase a topic name and still get a hit. + // A single hyphenated token is a literal string to copy, so extra + // positionals are an error rather than something to guess at. + if (positionals.length > 1) { + telemetry.capture("cli_help", { topic: positionals.join(" ") }); + console.error(`Too many arguments: ${positionals.join(" ")}`); + console.error( + "A topic is a single token. Run `taskless agent` for the topic index." + ); + process.exitCode = 1; + return; + } + + const key = positionals[0]!; // Anonymous variant lookup: prefer .anonymous.txt when // --anonymous is set, fall back to the canonical recipe. The lookup and - // the render both live in the shared prompts module, so `help` and the + // the render both live in the shared prompts module, so `agent` and the // `@taskless/cli/prompts` export emit the same text. const recipe = getRecipe(key, { anonymous: args.anonymous }); if (recipe) { // cli_help: agent fetched a specific recipe (intent signal). The topic // is the served topic; filtering on it replaces the old per-topic events. - telemetry.capture("cli_help", { topic: positionals.join(" ") }); + telemetry.capture("cli_help", { topic: key }); console.log(recipe.trimEnd()); } else { // cli_help for an unknown topic — still the attempted topic string. - telemetry.capture("cli_help", { topic: positionals.join(" ") }); - console.error(`Unknown command: ${positionals.join(" ")}`); - console.error("Run `taskless help` for available commands."); + telemetry.capture("cli_help", { topic: key }); + console.error(`Unknown command: ${key}`); + console.error("Run `taskless agent` for available topics."); process.exitCode = 1; } }, diff --git a/packages/cli/src/commands/rules.ts b/packages/cli/src/commands/rules.ts index 17250bb..301930c 100644 --- a/packages/cli/src/commands/rules.ts +++ b/packages/cli/src/commands/rules.ts @@ -105,7 +105,7 @@ const createCommand = defineCommand({ // Anonymous rule creation runs in the agent, not the CLI. Point the // agent at the local-only recipe and exit cleanly. const message = - "Anonymous rule generation runs in the agent. Run `taskless help rule create --anonymous` to fetch the local-only recipe."; + "Anonymous rule generation runs in the agent. Run `taskless agent rule-create --anonymous` to fetch the local-only recipe."; if (args.json) { console.log( JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message)) @@ -348,7 +348,7 @@ const improveCommand = defineCommand({ if (args.anonymous) { const message = - "Anonymous rule improvement runs in the agent. Run `taskless help rule improve --anonymous` to fetch the local-only recipe."; + "Anonymous rule improvement runs in the agent. Run `taskless agent rule-improve --anonymous` to fetch the local-only recipe."; if (args.json) { console.log( JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message)) diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 0ed1ba3..154bbe4 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -1,11 +1,11 @@ import { defineCommand, runCommand, showUsage } from "citty"; +import { createAgentCommand } from "./commands/agent"; import { authCommand } from "./commands/auth"; import { checkCommand } from "./commands/check"; import { detectCommand } from "./commands/detect"; import { initCommand, updateCommand } from "./commands/init"; import { infoCommand } from "./commands/info"; -import { createHelpCommand } from "./commands/help"; import { onboardCommand } from "./commands/onboard"; import { ruleCommand } from "./commands/rules"; import { @@ -27,7 +27,7 @@ const subCommands = { rule: ruleCommand, }; -const helpCommand = createHelpCommand(subCommands); +const agentCommand = createAgentCommand(subCommands); const main = defineCommand({ meta: { @@ -55,7 +55,7 @@ const main = defineCommand({ }, subCommands: { ...subCommands, - help: helpCommand, + agent: agentCommand, }, async run({ rawArgs, cmd }) { // citty always calls the parent's run handler, even after a subcommand. @@ -90,7 +90,7 @@ const main = defineCommand({ } // TTY → run the interactive wizard. Non-TTY → print a short preamble - // explaining the context and then delegate to `help` so agents and + // explaining the context and then delegate to `agent` so agents and // pipes see the topic index. if (process.stdout.isTTY === true && process.stdin.isTTY === true) { await runCommand(initCommand, { rawArgs }); @@ -101,11 +101,11 @@ const main = defineCommand({ "Taskless CLI — non-interactive context detected.\n" + " For interactive install, run from a terminal.\n" + " For scripted install, run `taskless init --no-interactive`.\n" + - " For agent recipes, run `taskless help` (no args) for the topic index.\n" + " For agent recipes, run `taskless agent` (no args) for the topic index.\n" ); - // Forward the parent's rawArgs (e.g. `-d `) so the help command + // Forward the parent's rawArgs (e.g. `-d `) so the agent command // doesn't mis-parse them as positional topic names. - await runCommand(helpCommand, { rawArgs: ["help", ...rawArgs] }); + await runCommand(agentCommand, { rawArgs: ["agent", ...rawArgs] }); }, }); diff --git a/packages/cli/src/telemetry-run.ts b/packages/cli/src/telemetry-run.ts index 3421b92..26607bf 100644 --- a/packages/cli/src/telemetry-run.ts +++ b/packages/cli/src/telemetry-run.ts @@ -7,8 +7,8 @@ import { CLIError } from "./util/cli-error"; * Derive the cli_run `command` property from the raw argv. Flags (and the * value after `-d`/`--dir`) are skipped; the first positional is the command, * and `rule` keeps its subcommand (e.g. `rule create`) since that distinction - * is meaningful. `help`'s topic is recorded separately on cli_help, so the - * command for a help invocation is just `help`. + * is meaningful. `agent`'s topic is recorded separately on cli_help, so the + * command for an agent invocation is just `agent`. */ export function resolveCommandName(rawArguments: string[]): string { const valueFlags = new Set(["-d", "--dir"]); diff --git a/packages/cli/test/anonymous-flag.test.ts b/packages/cli/test/anonymous-flag.test.ts index c763d4f..52a20f6 100644 --- a/packages/cli/test/anonymous-flag.test.ts +++ b/packages/cli/test/anonymous-flag.test.ts @@ -86,7 +86,7 @@ describe("--anonymous flag (per-command behavior matrix)", () => { // file validation. const result = await runCli(["rule", "create", "--anonymous", "-d", cwd]); expect(result.exitCode).not.toBe(0); - expect(result.stderr).toContain("taskless help rule create --anonymous"); + expect(result.stderr).toContain("taskless agent rule-create --anonymous"); }); it("with --json, emits the standardized envelope", async () => { @@ -115,7 +115,7 @@ describe("--anonymous flag (per-command behavior matrix)", () => { cwd, ]); expect(result.exitCode).not.toBe(0); - expect(result.stderr).toContain("taskless help rule improve --anonymous"); + expect(result.stderr).toContain("taskless agent rule-improve --anonymous"); }); }); diff --git a/packages/cli/test/cli-run.test.ts b/packages/cli/test/cli-run.test.ts index 40aa46b..c6a1ebe 100644 --- a/packages/cli/test/cli-run.test.ts +++ b/packages/cli/test/cli-run.test.ts @@ -9,7 +9,7 @@ describe("resolveCommandName", () => { [["check", "--json"], "check"], [["rule", "create"], "rule create"], [["rule"], "rule"], - [["help", "route"], "help"], + [["agent", "route"], "agent"], [["-d", "/tmp", "check"], "check"], [["--dir", "/tmp", "info"], "info"], [[], "(default)"], diff --git a/packages/cli/test/cli.test.ts b/packages/cli/test/cli.test.ts index 9291589..444c382 100644 --- a/packages/cli/test/cli.test.ts +++ b/packages/cli/test/cli.test.ts @@ -34,7 +34,7 @@ describe("cli", () => { }); describe("no args", () => { - it("shows help text and exits with code 0", async () => { + it("shows the agent topic index and exits with code 0", async () => { const { stdout } = await execFileAsync("node", [binPath]); expect(stdout).toContain("taskless"); expect(stdout).toContain("info"); diff --git a/packages/cli/test/help-extensions.test.ts b/packages/cli/test/help-extensions.test.ts index eca96b9..b715528 100644 --- a/packages/cli/test/help-extensions.test.ts +++ b/packages/cli/test/help-extensions.test.ts @@ -28,11 +28,11 @@ async function runCli( } } -describe("taskless help (no args)", () => { +describe("taskless agent (no args)", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-help-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-")); }); afterEach(async () => { @@ -40,14 +40,14 @@ describe("taskless help (no args)", () => { }); it("prints the human slug", async () => { - const result = await runCli(["help", "-d", cwd]); + const result = await runCli(["agent", "-d", cwd]); expect(result.exitCode).toBe(0); expect(result.stdout).toContain("For agents:"); expect(result.stdout).toContain("For humans:"); }); it("prints the topic table including all expected topics", async () => { - const result = await runCli(["help", "-d", cwd]); + const result = await runCli(["agent", "-d", cwd]); expect(result.stdout).toContain("Topics:"); for (const topic of ["init", "info", "check", "auth", "rule"]) { expect(result.stdout).toContain(topic); @@ -55,12 +55,12 @@ describe("taskless help (no args)", () => { }); it("mentions the --anonymous flag", async () => { - const result = await runCli(["help", "-d", cwd]); + const result = await runCli(["agent", "-d", cwd]); expect(result.stdout).toContain("--anonymous"); }); it("lists the routing recipe topics under Authoring recipes", async () => { - const result = await runCli(["help", "-d", cwd]); + const result = await runCli(["agent", "-d", cwd]); expect(result.stdout).toContain("Authoring recipes:"); for (const topic of [ "route", @@ -74,11 +74,11 @@ describe("taskless help (no args)", () => { }); }); -describe("taskless help ", () => { +describe("taskless agent ", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-help-routing-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-routing-")); }); afterEach(async () => { @@ -88,7 +88,7 @@ describe("taskless help ", () => { it.each(["route", "existing", "static", "remote", "engine-selection"])( "resolves the %s recipe without an unknown-topic error", async (topic) => { - const result = await runCli(["help", topic, "-d", cwd]); + const result = await runCli(["agent", topic, "-d", cwd]); expect(result.exitCode).toBe(0); expect(result.stdout).toContain(`# Topic: ${topic}`); expect(result.stdout).toContain("## Goal"); @@ -97,11 +97,11 @@ describe("taskless help ", () => { ); }); -describe("taskless help ", () => { +describe("taskless agent ", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-help-topic-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-topic-")); }); afterEach(async () => { @@ -109,7 +109,7 @@ describe("taskless help ", () => { }); it("returns the canonical recipe for a known topic", async () => { - const result = await runCli(["help", "rule", "create", "-d", cwd]); + const result = await runCli(["agent", "rule-create", "-d", cwd]); expect(result.exitCode).toBe(0); expect(result.stdout).toContain("# Topic: rule create"); expect(result.stdout).toContain("## Goal"); @@ -117,7 +117,7 @@ describe("taskless help ", () => { }); it("interpolates %(CLI_VERSION)s in the recipe header", async () => { - const result = await runCli(["help", "rule", "create", "-d", cwd]); + const result = await runCli(["agent", "rule-create", "-d", cwd]); // Should contain a version pattern, not the literal placeholder. // Guard against both the legacy mustache syntax and the current // sprintf-js named-arg syntax leaking through. @@ -127,7 +127,7 @@ describe("taskless help ", () => { }); it("interpolates %(INPUT_SCHEMA)s for topics with a Zod input", async () => { - const result = await runCli(["help", "rule", "create", "-d", cwd]); + const result = await runCli(["agent", "rule-create", "-d", cwd]); expect(result.stdout).not.toContain("{{INPUT_SCHEMA}}"); expect(result.stdout).not.toContain("%(INPUT_SCHEMA)s"); // Embedded schema includes the JSON Schema $schema URI @@ -137,7 +137,7 @@ describe("taskless help ", () => { }); it("renders %(PACKAGE_MANAGER_DLX)s as the agent-fill marker", async () => { - const result = await runCli(["help", "ci", "-d", cwd]); + const result = await runCli(["agent", "ci", "-d", cwd]); expect(result.exitCode).toBe(0); // Sprintf substitutes the placeholder; the agent-fill marker should // appear in its rendered form, never as the @@ -148,17 +148,34 @@ describe("taskless help ", () => { }); it("exits 1 for an unknown topic", async () => { - const result = await runCli(["help", "totally-unknown", "-d", cwd]); + const result = await runCli(["agent", "totally-unknown", "-d", cwd]); expect(result.exitCode).not.toBe(0); expect(result.stderr).toContain("Unknown command"); }); + + it("points an unknown topic at `taskless agent`, not the removed command", async () => { + const result = await runCli(["agent", "totally-unknown", "-d", cwd]); + expect(result.stderr).toContain("Run `taskless agent` for available"); + }); + + // A topic is one token. The old resolver joined positionals, so + // `rule create` and `create rule` both reached `rule-create.txt` — which + // invited an agent to paraphrase a topic name and still get a hit. Extra + // positionals are now an error rather than something to guess at. + it("rejects a multi-token topic instead of joining the positionals", async () => { + const result = await runCli(["agent", "rule", "create", "-d", cwd]); + expect(result.exitCode).not.toBe(0); + expect(result.stdout).not.toContain("# Topic:"); + expect(result.stderr).toContain("Too many arguments"); + expect(result.stderr).toContain("A topic is a single token"); + }); }); -describe("taskless help --anonymous (variant lookup)", () => { +describe("taskless agent --anonymous (variant lookup)", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-help-anon-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-anon-")); }); afterEach(async () => { @@ -167,9 +184,8 @@ describe("taskless help --anonymous (variant lookup)", () => { it("returns the .anonymous variant when one exists (rule create)", async () => { const result = await runCli([ - "help", - "rule", - "create", + "agent", + "rule-create", "--anonymous", "-d", cwd, @@ -181,9 +197,8 @@ describe("taskless help --anonymous (variant lookup)", () => { it("returns the .anonymous variant for rule improve", async () => { const result = await runCli([ - "help", - "rule", - "improve", + "agent", + "rule-improve", "--anonymous", "-d", cwd, @@ -193,24 +208,24 @@ describe("taskless help --anonymous (variant lookup)", () => { }); it("falls back to the canonical recipe when no variant exists (check)", async () => { - const canonical = await runCli(["help", "check", "-d", cwd]); - const anonymous = await runCli(["help", "check", "--anonymous", "-d", cwd]); + const canonical = await runCli(["agent", "check", "-d", cwd]); + const anonymous = await runCli(["agent", "check", "--anonymous", "-d", cwd]); expect(anonymous.exitCode).toBe(0); // Same body — falls back to check.txt since no check.anonymous.txt expect(anonymous.stdout).toBe(canonical.stdout); }); it("returns the canonical recipe when --anonymous is omitted", async () => { - const result = await runCli(["help", "rule", "create", "-d", cwd]); + const result = await runCli(["agent", "rule-create", "-d", cwd]); expect(result.stdout).toContain("# Topic: rule create"); expect(result.stdout).not.toContain("(anonymous)"); }); }); -describe("bare taskless (non-TTY) routes to help index", () => { +describe("bare taskless (non-TTY) routes to the agent index", () => { it("prints the non-interactive preamble + topic index", async () => { // execFile gives no TTY, which triggers the routing. No flags so - // citty doesn't try to forward them to the help subcommand. + // citty doesn't try to forward them to the agent subcommand. const result = await runCli([]); expect(result.exitCode).toBe(0); expect(result.stderr).toContain("non-interactive context detected"); @@ -218,11 +233,11 @@ describe("bare taskless (non-TTY) routes to help index", () => { }); }); -describe("taskless help engine-selection", () => { +describe("taskless agent engine-selection", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-help-engine-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-engine-")); }); afterEach(async () => { @@ -230,7 +245,7 @@ describe("taskless help engine-selection", () => { }); it("names all three engines and puts evidence before the answer", async () => { - const result = await runCli(["help", "engine-selection", "-d", cwd]); + const result = await runCli(["agent", "engine-selection", "-d", cwd]); expect(result.exitCode).toBe(0); for (const engine of ["`sg`", "`vale`", "`runtime`"]) { expect(result.stdout).toContain(engine); @@ -239,7 +254,7 @@ describe("taskless help engine-selection", () => { }); it("carries the three boundary cases", async () => { - const result = await runCli(["help", "engine-selection", "-d", cwd]); + const result = await runCli(["agent", "engine-selection", "-d", cwd]); // Each is a wrong answer someone actually reaches for. expect(result.stdout).toContain("Prose about code is still prose"); expect(result.stdout).toContain("one document at a time"); @@ -247,7 +262,7 @@ describe("taskless help engine-selection", () => { }); it("states the ambiguity default as a property, not as `sg`", async () => { - const result = await runCli(["help", "engine-selection", "-d", cwd]); + const result = await runCli(["agent", "engine-selection", "-d", cwd]); // D7: "choose an engine you know is available" stays correct on an // unsupported arch and server-side alike; naming `sg` outright would be // false on the host where `sg` is the missing one. @@ -256,13 +271,13 @@ describe("taskless help engine-selection", () => { }); it("stays out of the authoring-destination decision", async () => { - const result = await runCli(["help", "engine-selection", "-d", cwd]); + const result = await runCli(["agent", "engine-selection", "-d", cwd]); // Scope guard (3.4): it may point at `route`, never re-decide it. expect(result.stdout).toContain("is `route`, and it is a separate"); }); it("follows the recipe header and section convention", async () => { - const result = await runCli(["help", "engine-selection", "-d", cwd]); + const result = await runCli(["agent", "engine-selection", "-d", cwd]); expect(result.stdout).toContain("# Topic: engine-selection"); for (const section of [ "## Goal", diff --git a/packages/cli/test/help-routing-telemetry.test.ts b/packages/cli/test/help-routing-telemetry.test.ts index c3925b4..a391ae0 100644 --- a/packages/cli/test/help-routing-telemetry.test.ts +++ b/packages/cli/test/help-routing-telemetry.test.ts @@ -1,6 +1,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -// Spy on the telemetry capture by mocking the telemetry module the help +// Spy on the telemetry capture by mocking the telemetry module the agent // command imports. The factory is invoked lazily at import time, so the // closure over `capture` resolves after initialization (same pattern as // telemetry.test.ts mocking posthog-node). @@ -15,7 +15,7 @@ vi.mock("../src/telemetry", () => ({ shutdownTelemetry: () => Promise.resolve(), })); -const { createHelpCommand } = await import("../src/commands/help"); +const { createAgentCommand } = await import("../src/commands/agent"); interface RunnableCommand { run: (context: { @@ -24,7 +24,7 @@ interface RunnableCommand { }) => Promise; } -describe("help routing topics emit cli_help intent telemetry", () => { +describe("agent routing topics emit cli_help intent telemetry", () => { let logSpy: ReturnType; beforeEach(() => { @@ -40,10 +40,10 @@ describe("help routing topics emit cli_help intent telemetry", () => { it.each(["route", "existing", "static", "remote", "engine-selection"])( "captures cli_help for %s", async (topic) => { - const command = createHelpCommand({}) as unknown as RunnableCommand; + const command = createAgentCommand({}) as unknown as RunnableCommand; await command.run({ args: { dir: process.cwd(), anonymous: false }, - rawArgs: ["help", topic], + rawArgs: ["agent", topic], }); expect(capture).toHaveBeenCalledWith( diff --git a/packages/cli/test/help-telemetry.test.ts b/packages/cli/test/help-telemetry.test.ts index 5e11937..b103c23 100644 --- a/packages/cli/test/help-telemetry.test.ts +++ b/packages/cli/test/help-telemetry.test.ts @@ -3,7 +3,7 @@ import { join, resolve } from "node:path"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; -// Spy on telemetry by mocking the module the help command imports. The factory +// Spy on telemetry by mocking the module the agent command imports. The factory // is invoked lazily at import time (same pattern as telemetry.test.ts). const capture = vi.fn(); vi.mock("../src/telemetry", () => ({ @@ -13,7 +13,7 @@ vi.mock("../src/telemetry", () => ({ shutdownTelemetry: () => Promise.resolve(), })); -const { createHelpCommand } = await import("../src/commands/help"); +const { createAgentCommand } = await import("../src/commands/agent"); interface RunnableCommand { run: (context: { @@ -22,15 +22,15 @@ interface RunnableCommand { }) => Promise; } -async function runHelp(rawArguments: string[]): Promise { - const command = createHelpCommand({}) as unknown as RunnableCommand; +async function runAgent(rawArguments: string[]): Promise { + const command = createAgentCommand({}) as unknown as RunnableCommand; await command.run({ args: { dir: process.cwd(), anonymous: false }, rawArgs: rawArguments, }); } -describe("help emits cli_help { topic }", () => { +describe("agent emits cli_help { topic }", () => { let logSpy: ReturnType; let errorSpy: ReturnType; @@ -46,19 +46,27 @@ describe("help emits cli_help { topic }", () => { }); it("captures the served topic", async () => { - await runHelp(["help", "rule", "create"]); - expect(capture).toHaveBeenCalledWith("cli_help", { topic: "rule create" }); + await runAgent(["agent", "rule-create"]); + expect(capture).toHaveBeenCalledWith("cli_help", { topic: "rule-create" }); }); it("captures the index marker for no topic", async () => { - await runHelp(["help"]); + await runAgent(["agent"]); expect(capture).toHaveBeenCalledWith("cli_help", { topic: "(index)" }); }); it("captures the attempted topic for an unknown topic", async () => { - await runHelp(["help", "nope"]); + await runAgent(["agent", "nope"]); expect(capture).toHaveBeenCalledWith("cli_help", { topic: "nope" }); }); + + // The rejected invocation is still an intent signal — it says an agent + // reached for a topic by phrase rather than by token — so it is captured + // under the same event with the attempted words joined for readability. + it("captures the attempted words when too many positionals are given", async () => { + await runAgent(["agent", "rule", "create"]); + expect(capture).toHaveBeenCalledWith("cli_help", { topic: "rule create" }); + }); }); // Rather than asserting "no help_* event" inside every behavioral test above, diff --git a/packages/cli/test/onboard.test.ts b/packages/cli/test/onboard.test.ts index 298547d..c7afc25 100644 --- a/packages/cli/test/onboard.test.ts +++ b/packages/cli/test/onboard.test.ts @@ -182,7 +182,7 @@ describe("taskless onboard", () => { expect(stderr).toContain("--mark-complete"); }); - it("`taskless help onboard` matches the recipe printed by `taskless onboard --force`", async () => { + it("`taskless agent onboard` matches the recipe printed by `taskless onboard --force`", async () => { // Pre-mark onboarded so the `onboard` path also prints the recipe via // --force, ensuring we compare recipe-vs-recipe rather than gate-vs-recipe. await mkdir(join(cwd, ".taskless"), { recursive: true }); @@ -192,11 +192,11 @@ describe("taskless onboard", () => { "utf8" ); - const help = await runCli(["help", "onboard", "-d", cwd], cwd); + const agentRecipe = await runCli(["agent", "onboard", "-d", cwd], cwd); const onboard = await runCli(["onboard", "--force", "-d", cwd], cwd); - expect(help.exitCode).toBe(0); + expect(agentRecipe.exitCode).toBe(0); expect(onboard.exitCode).toBe(0); - expect(onboard.stdout.trim()).toBe(help.stdout.trim()); + expect(onboard.stdout.trim()).toBe(agentRecipe.stdout.trim()); }); }); diff --git a/packages/cli/test/prompts.test.ts b/packages/cli/test/prompts.test.ts index 8c35a76..f7de904 100644 --- a/packages/cli/test/prompts.test.ts +++ b/packages/cli/test/prompts.test.ts @@ -156,11 +156,11 @@ describe("typed accessor", () => { }); }); -describe("help command parity", () => { +describe("agent command parity", () => { it.each([...TOPICS])( - "matches `taskless help %s` byte for byte", + "matches `taskless agent %s` byte for byte", async (topic) => { - const { stdout } = await execFileAsync("node", [binPath, "help", topic]); + const { stdout } = await execFileAsync("node", [binPath, "agent", topic]); // The command trims trailing whitespace before printing; console.log then // adds the single newline that stdout carries. expect(stdout.trimEnd()).toBe(getPrompt(topic).trimEnd()); From 6ca8baffaa9391ce6e540ed5c195fbcfb4e9a87e Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 01:10:27 -0700 Subject: [PATCH 06/19] docs(openspec): add a harness that executes the authoring recipes The recipes are the deliverable of this change, and the failure mode is one review cannot see: prose that reads correctly to its author and produces the wrong artifact. So execute them instead of reading them. A fresh subagent gets the recipe text, a sandbox path, and a rule intent in plain words -- and no repository access, because with it the agent finds the existing `no-simply.yml` and copies it, and the loop tests our fixtures rather than our writing. Then check the artifacts mechanically and, the part that actually decides it, whether `check` reports the finding and `verify` passes. `build:dev` is the right target rather than `dist/`: it bakes an absolute path into the recipe text, so the agent reads the command it will really run. Against `dist/` we would be testing a recipe no reader receives. Three intents, not one. Everything in this repo today is `existence`, so a recipe drafted from our own examples teaches token blocklists -- and Vale has eleven extension points, most prose rules being substitutions or capitalization. `create-sg-rule` runs through the same harness as a control: it documents a flow that already works, so a failure there means the harness is wrong rather than the recipe. Converged when an agent, given an intent the recipe never names, produces a rule that fires on its fail fixture and stays quiet on its pass fixture first try. The iteration log is kept, because it is the only evidence a reviewer can check without rerunning the loop. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../agent-command-and-vale-authoring/tasks.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 1b3ec3a..2242056 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -22,6 +22,19 @@ - [ ] 2.7a Give every `create-*-rule` recipe the same opening orientation line: what topic this is, what it helps you write, and revisit routing if that is not what you need. Fixed shape across all five so an agent recognises it; scope only, never the comparison between engines - [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic +## 2b. Prove the authoring recipes by executing them + +The recipes are the deliverable, and a recipe that reads well to its author while producing the wrong artifact is exactly what reviewing the prose cannot catch. Execute them instead. + +- [ ] 2b.1 `pnpm --filter @taskless/cli build:dev`. This target exists for this: `TASKLESS_BUILD_TARGET=dev` bakes `__TASKLESS_CLI__` as an **absolute path** to `dist-dev/index.js`, so recipe text carries a command that actually runs from any directory. Testing against `dist/` instead would exercise a recipe no reader ever receives, since theirs says `npx @taskless/cli` +- [ ] 2b.2 Build the harness: scaffold a throwaway project in a temp directory using the built CLI, so the sandbox is a real `taskless init` scaffold — section-less `.vale.ini`, empty `vale/rules/` — and not a hand-made approximation of one +- [ ] 2b.3 Hand a **fresh, non-forked** subagent only three things: the recipe text, the sandbox path, and a rule intent stated in plain words. It must NOT have repository access. With it, the agent finds the existing `no-simply.yml` and the mixed-engine fixture and copies them, and the loop tests our fixtures rather than our writing +- [ ] 2b.4 Check the artifacts mechanically: a style file at `.taskless/vale/rules/.yml` with valid `extends`/`message`/`level`; a **scoped section** in `.vale.ini` enabling `rules.`; fixtures in both `pass/` and `fail/`. Then the assertion that matters — `check` reports the finding, and `verify` passes +- [ ] 2b.5 Iterate across three intents that exercise different extension points — one `existence`, one `substitution` (prefer X over Y), one `capitalization` (headings, product names). Everything in this repo today is `existence`, so a recipe drafted from our own examples teaches token blocklists and nothing else. Vale has eleven extension points and most real prose rules are not blocklists +- [ ] 2b.6 Every failure is a defect in the prose, not in the agent. Fix the recipe and re-run with a fresh agent. Converged when an agent, given an intent the recipe never names, produces a rule that fires on its `fail` fixture and stays quiet on its `pass` fixture, first try, uncorrected +- [ ] 2b.7 Keep the iteration log — what failed, what changed, what finally held. It is the evidence the prose works, and the only part of this a reviewer can check without rerunning the loop +- [ ] 2b.8 Run the same harness over `create-sg-rule` as a control. It documents a flow that already works, so a failure there means the harness is wrong rather than the recipe + ## 3. Sweep the cross-references - [ ] 3.1 Replace every `taskless help ` occurrence with `taskless agent ` across `src/help/*.txt`, `src/**/*.ts`, `skills/taskless/SKILL.md`, `README.md`, and `packages/cli/README.md` (~306 occurrences, 77 files). Leave `CHANGELOG.md` alone — it is a historical record @@ -44,7 +57,7 @@ ## 6. Verify - [ ] 6.1 `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli build`, `pnpm --filter @taskless/cli test` -- [ ] 6.2 **Rehearse the recipes against a fresh agent.** For `route` and each `create-*-rule`, hand the text to a subagent with no prior context and ask it to describe the steps it would take and the commands it would run. Check its account against what the recipe intends — a recipe that reads correctly to its author and produces the wrong plan is the failure mode prose review cannot catch. Feed the corrections back into the text +- [ ] 6.2 **Rehearse `route` against a fresh agent.** Hand it the text with no prior context and a request to author a rule, then check which destination it names and why. Unlike the authoring recipes (2b), `route` produces a decision rather than artifacts, so the plan it describes is the only thing there is to check. Cover one case per destination, including a logged-out runtime request - [ ] 6.2a Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero - [ ] 6.3 `pnpm openspec validate --all --strict` (note: `cli-rules` and `cli-update-engine` fail on `main` already and are unrelated) - [ ] 6.4 Archive the change From 1453142475d5028b6bcecf2519a24ca254bf10ed Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 08:51:08 -0700 Subject: [PATCH 07/19] docs(openspec): add resume notes for picking this change back up Handoff for a context reset. `tasks.md` is the authority on what is done; this records what a fresh reader cannot recover from the artifacts -- which worktree the work lives in, the environment traps that each cost an hour last session, the Vale facts already measured so they are not re-derived, and the landmines group 2 walks into. Points at the two decisions most likely to be re-litigated by accident: the engine criterion is stated once in `route` rather than copied into the destinations, and the section-less scaffold ships paired with stderr surfacing or it reintroduces the silent disable the Vale work exists to remove. Not part of the artifact set; delete it when the change is archived. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../resume.md | 111 ++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 openspec/changes/agent-command-and-vale-authoring/resume.md diff --git a/openspec/changes/agent-command-and-vale-authoring/resume.md b/openspec/changes/agent-command-and-vale-authoring/resume.md new file mode 100644 index 0000000..c07e65f --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/resume.md @@ -0,0 +1,111 @@ +# Resume notes — `agent-command-and-vale-authoring` + +Handoff for picking this up after a context reset. Not part of the OpenSpec artifact set; +delete it when the change is archived. + +## Where you are + +- **PR #102** (draft), branch `openspec/agent-command-and-vale-authoring`, stacked on **#100**. +- **Work in the worktree** at `worktrees/impl-102`, not the main checkout. The main checkout + should be on `main`; a branch can only be checked out in one worktree at a time. +- Stack below you, all green and already reviewed: **#71 → #93 → #94 → #95 → #100**. + The user drives the merge-down; do not merge anything without being asked. +- **Group 1 of 6 is complete** (`093aca1`). Everything else is unstarted. + `tasks.md` is the authority — read it first, and trust its checkboxes over this file. + +## Read these before doing anything + +1. `tasks.md` — the plan, including group **2b** (the recipe test harness) +2. `design.md` — decisions D1–D10, each with its rejected alternatives. All open questions closed. +3. `specs/*/spec.md` — what the recipes and the scaffold must do +4. `proposal.md` — the why, and the delivery shape (single PR, stacked on #100) + +## Environment traps that will cost you an hour each + +- **`NODE_OPTIONS` is broken in this session's shell.** It carries a `--require` preload + pointing at a deleted temp file, so every `node`, `pnpm`, and `git commit` (husky) call + dies with `MODULE_NOT_FOUND`. Prefix everything: + `NODE_OPTIONS="--max-old-space-size=4096"`. The symptom is a build that looks like it + exits 0 through a pipe while `dist/` is never written, which then fails ~125 unrelated tests. +- **Run `pnpm --filter @taskless/cli build` before `pnpm --filter @taskless/cli test`.** + Many suites spawn the built CLI. A stale or missing `dist/` produces failures that read + exactly like real regressions — this already caused one false alarm on `main`. +- **`commit.gpgsign` is true locally.** Keep it. An earlier restack silently stripped + signatures from 34 commits because `git rebase` does not re-sign without it, and CI does + not catch unsigned commits. If you rebase, audit with `git log --format='%G?'` afterwards. +- **Do not use `git checkout ` to undo a probe** when the file also holds uncommitted + work — it discards both. This bit twice in the previous session. + +## Next up: group 2, and how to do it + +Group 2 is the authorship — the part that needs care. The user wants to see the prose before +it is finalised, but **only after 2b has been run against it**, so they review a tested +artifact rather than a draft. + +Order that works: + +1. Draft `create-vale-rule.txt`. Source of truth for Vale specifics is + `https://docs.vale.sh/llms-full.txt` (a full corpus for LLM consumption), or + `https://docs.vale.sh/topics/.md?ask=`. **`https://docs.vale.sh/styles` 404s.** + Known-good facts already measured this session: + - Common rule fields: `extends` (required), `message` (required), + `level` (`suggestion`|`warning`|`error`, default `suggestion`), `scope`, `link`. + - Eleven extension points: `existence`, `substitution`, `occurrence`, `repetition`, + `consistency`, `conditional`, `capitalization`, `metric`, `spelling`, `sequence`, `script`. + - `StylesPath = .` makes `rules/` the StyleName, so a rule at `vale/rules/no-simply.yml` + has the check id `rules.no-simply`. `StylesPath = rules` resolves nothing — measured. + - A rule assignment outside any `[section]` is **silently ignored**; Vale reports + `W101 ... is ignoring it` on **stderr** with exit 0 and a valid `{}` on stdout. +2. Model its shape on `create-sg-rule.txt` (the renamed `static.txt`). If the Vale recipe + diverges structurally, that is a signal you invented something rather than followed the + established pattern. +3. **Run group 2b against it** before showing the user anything. See `tasks.md` 2b.1–2b.8. + The load-bearing constraint: the test subagent gets the recipe text, a sandbox path, and a + rule intent — and **no repository access**. With repo access it finds the existing + `no-simply.yml` and the mixed-engine fixture and copies them, and you are testing fixtures + rather than prose. +4. Bring the user the recipe **plus the iteration log**. + +## Group 2 landmines already identified + +- **`help-extensions.test.ts` has a five-test `taskless agent engine-selection` block.** + Task 2.7 deletes that topic, so those tests must be retired as part of the merge, not after. +- **`RECIPE_TOPICS` in `agent.ts`** still lists `route/existing/static/remote/engine-selection`. + Both `help-extensions.test.ts` and `help-routing-telemetry.test.ts` assert against that exact + list. Groups 2 and 4 both touch it. +- **The `remote` + `rule-create` merge (2.5a) is a content merge, not a rename.** Both texts + have material that survives; the result must read as one procedure rather than two + concatenated. This is the piece most worth the user's eye. +- Group 1 deliberately left recipe `.txt` files untouched. **Do not start sweeping + cross-references during group 2** — that is 3.1, and mixing them makes both unreviewable. + +## Group 3 note, when you get there + +`rule.txt` documents a table of multi-token forms (`taskless help rule create`, +`... rule meta`). After group 1 these are **actively broken**, not merely stale — they hit the +new "Too many arguments" path. Lead group 3 with it. + +## Decisions you should not silently revisit + +All are argued in `design.md` with rejected alternatives. The two most likely to be +re-litigated by accident: + +- **D1** — `route` and `engine-selection` merge; the engine criterion is stated **once**, in + `route`'s destination table. Destinations carry a short orientation line (D9), never a second + copy of the criterion. +- **D4** — the section-less scaffold ships **paired** with surfacing Vale's stderr on a + zero-exit run. Shipping the scaffold alone reintroduces the silent-disable failure the whole + Vale stack exists to eliminate. They are one requirement, not two. + +Also settled: pre-1.0, every backwards-incompatible change here is a **MINOR** bump — never +MAJOR. The telemetry event stays `cli_help` (the user wants agent-call volume visible under the +existing event). + +## Outside this PR + +- **#99** — migrate subprocess handling to execa (inventory and sequencing already written up) +- **#101** — whether a whole-project Vale check should skip build output; `.taskless/` is + already excluded as of #100 +- Stale worktrees under `worktrees/` from earlier agents; `git worktree list` to review +- `openspec validate --all --strict` fails on `spec/cli-rules` and `spec/cli-update-engine` + on `main` already — pre-existing, unrelated, do not chase it From b0a0ee029761004943a2b74ece380558b66c8eef Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 13:37:08 -0700 Subject: [PATCH 08/19] feat(cli): give every routed engine an authoring recipe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Group 2 of agent-command-and-vale-authoring: the recipes themselves. `route` absorbs `engine-selection` (D1). One fetch now goes from a rule-authoring request to a runnable command, with the engine criterion stated once, in route's destination table, and the runtime row split on login state (D7). Destinations open with a fixed orientation banner that self-corrects a misrouted reader without restating the criterion (D9). Topics flatten to verb-noun single tokens. `static` -> `create-sg-rule` also drops a leak: "static" is a trust tier, not an engine, and naming the ast-grep path after the tier taught the confusion the criterion exists to correct. `remote` + `rule-create` merge into `create-remote-rule` (D8); `rule-create.anonymous` is deleted rather than renamed, since it duplicated `static.txt` and a "local variant of the remote recipe" is the contradiction `route` resolves. Two recipes are new. `create-vale-rule` teaches the three artifacts, and above all the section: the scaffold ships section-less, so the first rule in a project also writes the first scope, and an assignment outside a section is ignored rather than rejected. `create-runtime-rule` is the logged-out path, explaining the gate as a property of executing code and deferring to `auth` (D6, D10). Groups 5.1 and 5.2 land here rather than later, deliberately: they are preconditions of testing the Vale recipe honestly. The scaffolded .vale.ini drops its `[*]`, and runVale surfaces a zero-exit stderr as a notice so the mistake that scaffold invites is legible instead of silent. Measured end to end — a top-level `rules. = YES` now prints Vale's W101 and still exits 0. Also corrects three spec deltas that still referenced `engine-selection` as a live topic, and re-homes the two engine-reasoning requirements that survive the merge onto `route`. 572 tests pass; typecheck and lint clean. --- .../specs/cli-help/spec.md | 17 +- .../specs/cli-knowledge-prompts/spec.md | 14 +- .../specs/cli-rule-routing/spec.md | 27 +++ .../agent-command-and-vale-authoring/tasks.md | 27 ++- packages/cli/src/commands/agent.ts | 13 +- packages/cli/src/commands/rules.ts | 9 +- .../filesystem/migrations/0004-vale-engine.ts | 12 +- .../{existing.txt => create-legacy-rule.txt} | 15 +- packages/cli/src/help/create-remote-rule.txt | 141 ++++++++++++ packages/cli/src/help/create-runtime-rule.txt | 118 ++++++++++ packages/cli/src/help/create-sg-rule.txt | 113 +++++++++ packages/cli/src/help/create-vale-rule.txt | 200 ++++++++++++++++ .../help/{rule-delete.txt => delete-rule.txt} | 0 packages/cli/src/help/engine-selection.txt | 113 --------- ...onymous.txt => improve-rule.anonymous.txt} | 0 .../{rule-improve.txt => improve-rule.txt} | 0 packages/cli/src/help/remote.txt | 60 ----- packages/cli/src/help/route.txt | 217 ++++++++++++------ .../cli/src/help/rule-create.anonymous.txt | 112 --------- packages/cli/src/help/rule-create.txt | 101 -------- packages/cli/src/help/static.txt | 76 ------ .../help/{rule-verify.txt => verify-rule.txt} | 0 packages/cli/src/prompts/index.ts | 45 ++-- packages/cli/src/prompts/recipes.ts | 4 +- packages/cli/src/rules/dispatch.ts | 10 +- packages/cli/src/rules/vale/run.ts | 29 ++- packages/cli/test/anonymous-flag.test.ts | 4 +- packages/cli/test/help-extensions.test.ts | 167 ++++++++++---- packages/cli/test/prompts.test.ts | 42 ++-- 29 files changed, 1030 insertions(+), 656 deletions(-) rename packages/cli/src/help/{existing.txt => create-legacy-rule.txt} (78%) create mode 100644 packages/cli/src/help/create-remote-rule.txt create mode 100644 packages/cli/src/help/create-runtime-rule.txt create mode 100644 packages/cli/src/help/create-sg-rule.txt create mode 100644 packages/cli/src/help/create-vale-rule.txt rename packages/cli/src/help/{rule-delete.txt => delete-rule.txt} (100%) delete mode 100644 packages/cli/src/help/engine-selection.txt rename packages/cli/src/help/{rule-improve.anonymous.txt => improve-rule.anonymous.txt} (100%) rename packages/cli/src/help/{rule-improve.txt => improve-rule.txt} (100%) delete mode 100644 packages/cli/src/help/remote.txt delete mode 100644 packages/cli/src/help/rule-create.anonymous.txt delete mode 100644 packages/cli/src/help/rule-create.txt delete mode 100644 packages/cli/src/help/static.txt rename packages/cli/src/help/{rule-verify.txt => verify-rule.txt} (100%) diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md index 72a1d57..268167a 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md @@ -33,20 +33,23 @@ The subcommand is named for its reader. It serves agents fetching a procedure, n - **WHEN** a user runs `taskless help check` - **THEN** the CLI SHALL NOT print recipe text for `check` -### Requirement: Routing recipes reference engine selection +### Requirement: Routing recipes name a destination, not a second decision -The `route` recipe SHALL apply the engine-selection reasoning directly and name a concrete authoring topic, rather than referring the reader onward to select an engine. The `create-sg-rule` and `create-vale-rule` recipes SHALL reference `engine-selection` so an agent that arrives at one directly can confirm the engine is right for the rule in hand. +The `route` recipe SHALL apply the engine reasoning directly and name a concrete `create-*-rule` topic, rather than referring the reader onward to a topic that selects an engine. No shipped recipe SHALL refer to `engine-selection`, which no longer exists. + +Each `create-*-rule` recipe SHALL instead point back at `route` for a reader who arrived at the wrong one, so recovery costs a re-decision rather than a second copy of the criterion (see "Every authoring recipe opens by orienting the reader"). #### Scenario: Route names a destination without a second fetch - **WHEN** an agent follows `route` -- **THEN** the recipe SHALL name one `create-*-rule` topic or `remote` -- **AND** it SHALL NOT require fetching `engine-selection` first to do so +- **THEN** the recipe SHALL name one `create-*-rule` topic +- **AND** it SHALL NOT require fetching a separate engine-selection topic first to do so -#### Scenario: Authoring recipes cross-reference engine selection +#### Scenario: Authoring recipes point back rather than re-deciding -- **WHEN** an agent reads `create-sg-rule` or `create-vale-rule` -- **THEN** the recipe SHALL reference `taskless agent engine-selection` as the check that the engine matches the rule +- **WHEN** an agent reads any `create-*-rule` recipe +- **THEN** the recipe SHALL name `route` as where to go if this is the wrong destination +- **AND** it SHALL NOT reference `engine-selection` ## ADDED Requirements diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md index fce97a1..4722b08 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md @@ -22,18 +22,20 @@ What the requirement actually protects is not the version number but the notice. ## ADDED Requirements -### Requirement: Exported topics cover both static-tier authoring paths +### Requirement: Exported topics cover every engine a rule can be routed to -`TOPICS` SHALL export the authoring recipe for each static-tier engine — `create-sg-rule` and `create-vale-rule` — alongside `engine-selection`. +`TOPICS` SHALL export the authoring recipe for each engine — `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule`. -A consumer that can decide a rule belongs to `vale` must be able to reach the procedure for authoring one. Exporting the chooser without the Vale destination reproduces, for the platform generator, the dead end this change removes from the CLI. +A consumer that can decide a rule belongs to an engine must be able to reach the procedure for authoring one. Exporting a chooser without its destinations reproduces, for the platform generator, the dead end this change removes from the CLI. -#### Scenario: Both authoring paths are reachable from the export +`engine-selection` leaves the export because it stops existing: the criterion it carried now lives in `route`, stated once. `route` is not exported here — it still contains local mechanics a Worker cannot run — so until it is, a consumer gets each destination's own scope from these three and adjudicates a genuinely ambiguous call itself. + +#### Scenario: Every engine's authoring path is reachable from the export - **WHEN** a consumer imports `TOPICS` -- **THEN** it SHALL contain `create-sg-rule`, `create-vale-rule`, and `engine-selection` +- **THEN** it SHALL contain `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule` #### Scenario: The exported set follows the rename - **WHEN** a consumer imports `TOPICS` -- **THEN** it SHALL NOT contain `static`, which no longer names a recipe +- **THEN** it SHALL NOT contain `static` or `engine-selection`, neither of which names a recipe any more diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md index 85c90f6..a395e83 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md +++ b/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md @@ -64,6 +64,33 @@ The recipe SHALL be named for the artifact it produces rather than for a trust t - **THEN** it SHALL write the rule on-device without requiring login or the Taskless API +### Requirement: Available code context outranks the phrasing of the request + +Where code or diff context is available, `route` SHALL weigh the concrete syntactic form present in the repository above the wording of the request, since the same request routes differently depending on the form the code actually takes. + +This bound the standalone engine-selection topic. That topic is gone, but the reasoning is not — it now binds the place the decision is actually made. + +#### Scenario: Concrete form changes the engine + +- **WHEN** a rule is statically correlatable in the form the repository actually contains +- **THEN** `route` selects `create-sg-rule` +- **AND WHEN** the equivalent rule requires normalizing a captured value to match a declaration elsewhere +- **THEN** it selects a runtime destination, despite an identically phrased request + +### Requirement: Ambiguity resolves to an engine known to be available + +When no engine is clearly indicated, `route` SHALL direct the reader to choose an engine whose availability can be asserted in the situation at hand, and to give that availability as the reason for the call. It SHALL NOT name a fixed fallback engine. Both `sg` and `vale` ship as platform binaries, so either can be the missing one on an unsupported architecture or where an install was blocked; server-side the constraint is different again, `sg` being the only ungated route. A named default is wrong in whichever of those situations it failed to anticipate, which is why the requirement is stated as a property rather than as a fact about any one engine. + +#### Scenario: Ambiguous request resolves to an assertably available engine + +- **WHEN** the available context does not disambiguate which engine can enforce a rule +- **THEN** `route` selects an engine whose availability it can assert, and states that availability as the reasoning that made the call close + +#### Scenario: The default is never an unavailable engine + +- **WHEN** an engine is unavailable in the current environment, such as the Vale binary being absent +- **THEN** the ambiguity default SHALL NOT name it + ### Requirement: Trust tier is not an engine-selection input Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to the engine choice: `sg` and `vale` are both static-tier, and only `runtime` carries those concerns, so trust tier is a distinct axis from which engine can express a rule. diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 2242056..96e1e90 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -11,15 +11,19 @@ ## 2. Rename and add the authoring topics -- [ ] 2.1 `git mv` `help/static.txt` → `help/create-sg-rule.txt`; retitle its header and rewrite its Goal to name the artifact rather than the tier -- [ ] 2.2 `git mv` `help/existing.txt` → `help/create-legacy-rule.txt`; retitle and update its header -- [ ] 2.3 Flatten the `rule-*` topics to verb-noun single tokens (`rule-create` → `create-rule`, `rule-improve` → `improve-rule`, `rule-delete` → `delete-rule`, `rule-verify` → `verify-rule`), including their `.anonymous` variants. Decide `rule-meta` and `rule` deliberately — they are not creation verbs and may keep their names -- [ ] 2.4 Author `help/create-vale-rule.txt`: the three artifacts (style file, `.vale.ini` section, `pass/`/`fail` fixtures), that the scaffold ships section-less so the first rule writes the first scope, and that a rule enabled outside a section is ignored by Vale. State the evidence that makes `vale` the right engine for a rule, since no chooser topic states it any more. Cross-reference `verify-rule` -- [ ] 2.5 Author `help/create-runtime-rule.txt` as the logged-**out** path: what a runtime rule is, where its `check.ts` lives, and why executing code requires login, reconciliation, and signing when the static tiers do not. Point at `auth` for obtaining access rather than restating it. It must not forward to another authoring recipe -- [ ] 2.5a Merge `help/remote.txt` and `help/rule-create.txt` into `help/create-remote-rule.txt` (plus the `.anonymous` variant). A content merge, not a rename: both texts have material that survives, and the result must read as one procedure rather than two concatenated -- [ ] 2.6 Rewrite `help/route.txt` to read login state early and classify into the five `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Offer `create-remote-rule` only where it is a genuine choice — locally expressible AND logged in. A logged-in runtime request routes straight to `create-remote-rule`; a logged-out one to `create-runtime-rule`. It must name a command the agent can run verbatim -- [ ] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`'s destination table — stated once, not copied into the destinations -- [ ] 2.7a Give every `create-*-rule` recipe the same opening orientation line: what topic this is, what it helps you write, and revisit routing if that is not what you need. Fixed shape across all five so an agent recognises it; scope only, never the comparison between engines +- [x] 2.1 `git mv` `help/static.txt` → `help/create-sg-rule.txt`; retitle its header and rewrite its Goal to name the artifact rather than the tier + - Also folded in the material from `rule-create.anonymous.txt` that `static.txt` lacked: the upstream-schema pointer, the optional-field list, and the per-layer verify error table +- [x] 2.2 `git mv` `help/existing.txt` → `help/create-legacy-rule.txt`; retitle and update its header +- [x] 2.3 Flatten the `rule-*` topics to verb-noun single tokens (`rule-create` → `create-rule`, `rule-improve` → `improve-rule`, `rule-delete` → `delete-rule`, `rule-verify` → `verify-rule`), including their `.anonymous` variants. Decide `rule-meta` and `rule` deliberately — they are not creation verbs and may keep their names + - **`rule-create` does not become `create-rule`.** 2.5a consumes it: the API-backed text *is* the service procedure, so it became `create-remote-rule`. A `create-rule` topic would have been a third name for the same thing + - **`rule-create.anonymous.txt` is deleted, not renamed.** It duplicated `static.txt` — both are "author an ast-grep rule locally, no service call" — and carrying it forward as `create-remote-rule.anonymous.txt` would have meant "the local variant of the remote recipe", which is the contradiction `route` exists to resolve. Its unique material moved into `create-sg-rule.txt` (2.1), and `rule create --anonymous` now points there. This is the one place group 2 deviates from the task text as written + - **`rule-meta` and `rule` keep their names.** Neither is a creation verb, both are already single tokens, and `rule`'s broken table is group 3's opening item +- [x] 2.4 Author `help/create-vale-rule.txt`: the three artifacts (style file, `.vale.ini` section, `pass/`/`fail` fixtures), that the scaffold ships section-less so the first rule writes the first scope, and that a rule enabled outside a section is ignored by Vale. State the evidence that makes `vale` the right engine for a rule, since no chooser topic states it any more. Cross-reference `verify-rule` +- [x] 2.5 Author `help/create-runtime-rule.txt` as the logged-**out** path: what a runtime rule is, where its `check.ts` lives, and why executing code requires login, reconciliation, and signing when the static tiers do not. Point at `auth` for obtaining access rather than restating it. It must not forward to another authoring recipe +- [x] 2.5a Merge `help/remote.txt` and `help/rule-create.txt` into `help/create-remote-rule.txt` (no `.anonymous` variant — see 2.3). A content merge, not a rename: both texts have material that survives, and the result must read as one procedure rather than two concatenated +- [x] 2.6 Rewrite `help/route.txt` to read login state early and classify into the five `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Offer `create-remote-rule` only where it is a genuine choice — locally expressible AND logged in. A logged-in runtime request routes straight to `create-remote-rule`; a logged-out one to `create-runtime-rule`. It must name a command the agent can run verbatim +- [x] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`'s destination table — stated once, not copied into the destinations +- [x] 2.7a Give every `create-*-rule` recipe the same opening orientation line: what topic this is, what it helps you write, and revisit routing if that is not what you need. Fixed shape across all five so an agent recognises it; scope only, never the comparison between engines - [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic ## 2b. Prove the authoring recipes by executing them @@ -49,8 +53,9 @@ The recipes are the deliverable, and a recipe that reads well to its author whil ## 5. Scaffold and diagnostics (ships together) -- [ ] 5.1 `VALE_CONFIG_CONTENT` in `0004-vale-engine.ts` drops its `[*]` section, leaving `StylesPath` and `MinAlertLevel` -- [ ] 5.2 `runVale` captures stderr on a zero-exit run and returns it as a notice on the `ok` outcome; `runValeEngine` forwards it to `DispatchResult.notices`. A notice must not touch the exit code +- [x] 5.1 `VALE_CONFIG_CONTENT` in `0004-vale-engine.ts` drops its `[*]` section, leaving `StylesPath` and `MinAlertLevel` +- [x] 5.2 `runVale` captures stderr on a zero-exit run and returns it as a notice on the `ok` outcome; `runValeEngine` forwards it to `DispatchResult.notices`. A notice must not touch the exit code + - **Pulled forward out of order, deliberately.** 2b tests `create-vale-rule` against a scaffolded project, and the recipe's central claim is that the scaffold ships section-less. Running the harness against a scaffold that still wrote `[*]` would have tested a recipe nobody will receive. Measured end to end afterwards: section-less scaffold + a top-level `rules. = YES` now prints `Notice: Vale reported while running: W101 'rules.no-simply' isn't a core option; Vale is ignoring it.` and exits 0 - [ ] 5.3 Test that a rule enabled outside a section produces a notice containing Vale's `W101` text and exits zero — this is the pairing that keeps 5.1 from reintroducing a silent disable - [ ] 5.4 Extend the mixed-engine integration test: a scaffolded project with a rule file and no section reports nothing and does not fail; adding a section makes the same rule fire diff --git a/packages/cli/src/commands/agent.ts b/packages/cli/src/commands/agent.ts index e50647b..da79afa 100644 --- a/packages/cli/src/commands/agent.ts +++ b/packages/cli/src/commands/agent.ts @@ -14,11 +14,12 @@ import { getRecipe } from "../prompts/recipes"; // discoverable from the `taskless agent` index. The rule-authoring front // door (`route`) and its destinations live here so an agent can find them. const RECIPE_TOPICS: ReadonlyArray<[string, string]> = [ - ["route", "Decide where to author a rule (existing/static/remote)"], - ["existing", "Author a rule in a linter the repo already uses"], - ["static", "Author a local ast-grep rule on this machine (no login)"], - ["remote", "Generate a rule via the Taskless service (login)"], - ["engine-selection", "Decide which engine enforces a rule (sg/vale/runtime)"], + ["route", "Decide which recipe authors a rule (start here)"], + ["create-legacy-rule", "Author a rule in a linter the repo already uses"], + ["create-sg-rule", "Author a local ast-grep rule over code (no login)"], + ["create-vale-rule", "Author a local Vale rule over prose (no login)"], + ["create-runtime-rule", "The runtime tier, and why it needs an account"], + ["create-remote-rule", "Generate a rule via the Taskless service (login)"], ]; async function unwrap(resolvable: Resolvable): Promise { @@ -119,7 +120,7 @@ export function createAgentCommand(subCommands: SubCommandsDef) { ); console.log("and use local-only behavior."); console.log( - "\nRun `taskless agent ` for the full recipe (e.g. `taskless agent rule-create`)." + "\nRun `taskless agent ` for the full recipe (e.g. `taskless agent create-sg-rule`)." ); return; } diff --git a/packages/cli/src/commands/rules.ts b/packages/cli/src/commands/rules.ts index 301930c..2033927 100644 --- a/packages/cli/src/commands/rules.ts +++ b/packages/cli/src/commands/rules.ts @@ -103,9 +103,12 @@ const createCommand = defineCommand({ if (args.anonymous) { // Anonymous rule creation runs in the agent, not the CLI. Point the - // agent at the local-only recipe and exit cleanly. + // agent at the local-only recipe and exit cleanly. That recipe is + // `create-sg-rule`: authoring an ast-grep rule on-device with no service + // call is exactly what anonymous mode asks for, so it is the destination + // rather than an `--anonymous` variant of the service recipe. const message = - "Anonymous rule generation runs in the agent. Run `taskless agent rule-create --anonymous` to fetch the local-only recipe."; + "Anonymous rule generation runs in the agent. Run `taskless agent create-sg-rule` to fetch the local-only recipe."; if (args.json) { console.log( JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message)) @@ -348,7 +351,7 @@ const improveCommand = defineCommand({ if (args.anonymous) { const message = - "Anonymous rule improvement runs in the agent. Run `taskless agent rule-improve --anonymous` to fetch the local-only recipe."; + "Anonymous rule improvement runs in the agent. Run `taskless agent improve-rule --anonymous` to fetch the local-only recipe."; if (args.json) { console.log( JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message)) diff --git a/packages/cli/src/filesystem/migrations/0004-vale-engine.ts b/packages/cli/src/filesystem/migrations/0004-vale-engine.ts index 6d7baab..fe3c9b8 100644 --- a/packages/cli/src/filesystem/migrations/0004-vale-engine.ts +++ b/packages/cli/src/filesystem/migrations/0004-vale-engine.ts @@ -34,8 +34,18 @@ const SG_CONFIG_CONTENT = `ruleDirs:\n - rules\ntestConfigs:\n - testDir: rule * nothing, Vale reports `{}`, and a prose check passes clean with every rule * silently disabled. Measured against the real binary, which is the only way * this is visible — the layout is identical either way. + * + * It carries **no section**, so a scaffolded project lints nothing until an + * author scopes something deliberately. An unscoped `[*]` would apply every + * enabled rule to every file the walk reaches, making the default the widest + * scope available rather than the narrowest — and scope is the author's + * decision to make. `create-vale-rule` teaches writing the first section; the + * mistake that invites (a `rules. = YES` above the first `[…]` line, which + * Vale ignores with a `W101` on stderr and exit 0) is why `runVale` surfaces a + * zero-exit stderr as a notice. The two ship together: without the notice, this + * scaffold would trade a too-wide default for a silent one. */ -const VALE_CONFIG_CONTENT = `StylesPath = .\nMinAlertLevel = suggestion\n\n[*]\n`; +const VALE_CONFIG_CONTENT = `StylesPath = .\nMinAlertLevel = suggestion\n`; /** Directories that must exist after the migration, tracked when empty. */ const SCAFFOLD_DIRECTORIES = [ diff --git a/packages/cli/src/help/existing.txt b/packages/cli/src/help/create-legacy-rule.txt similarity index 78% rename from packages/cli/src/help/existing.txt rename to packages/cli/src/help/create-legacy-rule.txt index efc00e5..6920cc8 100644 --- a/packages/cli/src/help/existing.txt +++ b/packages/cli/src/help/create-legacy-rule.txt @@ -1,4 +1,11 @@ -# Topic: existing (CLI v%(CLI_VERSION)s / topic v1) +# Topic: create-legacy-rule (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `create-legacy-rule`. It helps you write a rule for a linter the +repository already runs — ESLint, Ruff, RuboCop, Stylelint — in that +tool's own dialect, so that tool enforces it. +If that is not the kind of check you need, re-run `taskless agent route` +and follow its decision rather than adapting this recipe. ## Goal Author a rule in a linter the repository ALREADY uses, expressed in that @@ -52,6 +59,6 @@ then write the rule where that tool expects it. ## See Also -- `taskless help route` — re-decide the destination if this no longer fits -- `taskless help static` — author a local ast-grep rule instead -- `taskless help remote` — generate via the Taskless service (login) +- `taskless agent route` — re-decide the destination if this no longer fits +- `taskless agent create-sg-rule` — author a local ast-grep rule instead +- `taskless agent create-remote-rule` — generate via the service (login) diff --git a/packages/cli/src/help/create-remote-rule.txt b/packages/cli/src/help/create-remote-rule.txt new file mode 100644 index 0000000..24c6fdb --- /dev/null +++ b/packages/cli/src/help/create-remote-rule.txt @@ -0,0 +1,141 @@ +# Topic: create-remote-rule (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `create-remote-rule`. It helps you have the Taskless service +write a rule, when the rule is beyond what you can express on-device or +the user has chosen to spend a generation on it. +If that is not the kind of check you need, re-run `taskless agent route` +and follow its decision rather than adapting this recipe. + +## Goal +Enrich the user's description into a request the service can act on, +submit it, and report what came back. The service generates the rule and +writes the rule files; your job is everything on either side of that +call. + +This recipe is the whole path — the boundary and the procedure. Do not +go looking for a second topic to perform the submission. + +## Preconditions +- `.taskless/` directory exists. +- The user is logged in. This path requires auth. +- The repository has a GitHub origin remote. +- The request reached here through `route`, or through a local attempt + that failed and a user who confirmed the escalation. + +## Cost, and when this path is right + +A generation costs the user something and requires an account. That is +why `route` prefers a local rule it can write and verify, and why this +recipe is not a default. + +Two ways to legitimately be here: + +- **`route` sent you.** Either the rule is not expressible on-device, or + it is expressible and the user was offered the choice and took it. +- **A local attempt failed and the user confirmed.** If you came from + `create-sg-rule`'s failure path, that recipe has already deleted the + broken candidate and asked. If it has not been asked, ask now — a + silent fall-through from a failed local attempt to a paid service call + is not acceptable, even when the service would obviously do better. + +## Steps + +1. **Confirm auth.** Run: + ``` + npx @taskless/cli info --json + ``` + Check `loggedIn`. If false, fetch `taskless agent auth` and follow the + login recipe before continuing. Do not build a request you cannot + submit. + +2. **Check for a rule that already covers this.** Scan + `.taskless/sg/rules/` and read each rule's `message`, `note`, and + `rule` fields. If one overlaps, show the user and ask whether they + would rather iterate on it — `taskless agent improve-rule` refines an + existing rule and is usually the better answer than a second rule + that half-overlaps the first. + +3. **Gather the request.** Even when the user supplied a description, + ask: + - What exact pattern should be flagged? Get concrete examples. + - In what language? + - Where is the pattern acceptable — are there contexts to exclude? + + If you arrived from a failed local attempt, reuse the cases you + already gathered rather than asking again. + +4. **Enrich it.** This is the part that decides the quality of what + comes back: + - Search the codebase for real instances and show the user what you + found. Real matches beat invented ones. + - Ask for further success and failure cases, especially near-misses. + - Read `.gitignore`, the linter configs, and `tsconfig.json`'s + `exclude` for exclusion patterns the repo already believes in. + Present them as proposed defaults rather than applying them. + - Infer the primary language from the codebase and confirm it. Put it + in the `prompt` field. + +5. **Confirm before spending the generation.** Summarize the full + prompt, the success cases, the failure cases, and the exclusions. + This is the last point at which a misunderstanding is free. + +6. **Write the payload** to `.taskless/.tmp-rule-request.json`, matching + the input schema below. + +7. **Submit.** Run: + ``` + npx @taskless/cli rule create --from .taskless/.tmp-rule-request.json --json + ``` + This may take 30–60 seconds while the service generates the rule. + +8. **Clean up.** Delete `.taskless/.tmp-rule-request.json` whether the + call succeeded or failed. + +9. **Report.** The service writes the rule to `.taskless/sg/rules/.yml`, + its tests to `.taskless/sg/rule-tests/-YYYYMMDD-test.yml`, and + metadata to `.taskless/rule-metadata/.yml`. These are the same + paths and the same shape a locally authored rule uses, so `check`, + `improve-rule`, and `verify-rule` treat them identically. Show the + user the paths and suggest `taskless agent check`. + +## Input schema + +The `--from` JSON file conforms to: + +```json +%(INPUT_SCHEMA)s +``` + +Each entry in `successCases` and `failureCases` is a separate string. +Multi-line code goes in one string with literal newlines. + +## Important Notes + +- Do NOT pre-build the rule yourself here. Submit the request and let + the service generate it; a hand-written rule attached to a generation + request is neither reviewed nor used. +- The service owns rule-type selection. Today it generates ast-grep + rules under `.taskless/sg/rules/` — the same shape `create-sg-rule` + produces locally. + +## Errors + +With `--json`, failures emit `{ ok: false, code, message }`: + +| code | meaning | fix | +|--------------------------|------------------------------------|----------------------------------------------| +| `AUTH_REQUIRED` | not logged in | fetch `taskless agent auth` | +| `NO_GITHUB_REMOTE` | no GitHub origin remote | tell the user; we cannot proceed | +| `INVALID_INPUT` | `--from` JSON failed validation | re-read the input schema, fix, retry | +| `NETWORK_ERROR` | submit/poll failed | report and suggest retry | +| `RULE_GENERATION_FAILED` | the service failed to generate | report the message; suggest enriching prompt | +| `RULE_UNSUPPORTED` | plan lacks this generation type | tell the user to enable it; do not retry | + +## See Also + +- `taskless agent route` — the routing decision that leads here +- `taskless agent auth` — log in before generating +- `taskless agent create-sg-rule` — author a rule locally instead +- `taskless agent improve-rule` — iterate on a rule that already exists +- `taskless agent check` — validate the generated rule diff --git a/packages/cli/src/help/create-runtime-rule.txt b/packages/cli/src/help/create-runtime-rule.txt new file mode 100644 index 0000000..2445dd2 --- /dev/null +++ b/packages/cli/src/help/create-runtime-rule.txt @@ -0,0 +1,118 @@ +# Topic: create-runtime-rule (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `create-runtime-rule`. It helps you write a runtime rule: a +check that runs your own code, because answering it needs more than one +file — the repository graph, git metadata, build output, a resolved +config chain. +If that is not the kind of check you need, re-run `taskless agent route` +and follow its decision rather than adapting this recipe. + +You are reading this recipe rather than `create-remote-rule` because the +user is **not logged in**. That is the whole difference, and this recipe +explains it once. + +## Goal +Explain what a runtime rule is, why this tier is gated when the static +tiers are not, and what the user has to do before one can run. + +## Preconditions +- `.taskless/` directory exists. +- The user is **not** logged in. If `taskless info --json` reports + `loggedIn: true`, you are in the wrong recipe — re-run + `taskless agent route`. + +## What a runtime rule is + +A rule directory under `.taskless/runtime/rules//` holding two +kinds of file: + +| File | Role | +|---------------|------------------------------------------------------------------| +| `*.yml` | ast-grep capture rules that narrow which files the check looks at | +| `check.ts` | a module whose default export receives those matches and returns findings | + +The capture rules do the cheap work — if nothing matches, `check.ts` is +never invoked. `check.ts` does the part no static engine can: hold two +files at once, read git, resolve an alias, look outside the repository. + +That is also exactly why it is gated. + +## Why this tier is gated + +**Because it executes code, not because of what it can express.** + +`sg` and `vale` rules are inert data. Whatever is in them, the worst a +malicious rule achieves is a wrong finding — `taskless check` runs them +with no login, no network, and nothing to verify, because there is +nothing to verify. + +A runtime rule's `check.ts` is a program that runs on the developer's +machine with the developer's permissions. A rule file that arrives in a +pull request is code that arrives in a pull request. So `check`: + +1. **Signs** each `check.ts` — a signature over the exact bytes on disk. +2. **Reconciles** those signatures with the Taskless service, which + answers with the set it will vouch for. +3. **Runs only what came back blessed.** Anything else is reported as + skipped, with the reason. + +Reconciliation is what makes the signature mean something: it is the +service saying "this rule, these exact bytes, for this repository." +Without a login there is nobody to ask, so there is no answer, so +nothing runs. + +This is a property of executing code. It says nothing about whether a +runtime rule is more or less capable, and it is not a quality tier — +`sg` and `vale` rules are not less trusted, they are unexecuted. + +## What happens today, logged out + +Nothing breaks. `taskless check` still runs every static rule; each +runtime rule it finds is listed as skipped with the reason +`not authenticated — runtime rules were not verified and did not run`. + +So a runtime rule you write now is inert until the user logs in. Say +that plainly rather than letting them discover it from a silent check. + +## Steps + +1. **Tell the user what their rule needs, and why it is gated.** Name + the evidence — "this has to compare two files", "this reads git + history" — and then the consequence: it has to run code, so it needs + an account. Do not present this as a limitation of the rule. + +2. **Point them at login.** Obtaining access is `auth`'s job, and this + recipe deliberately does not restate it: + + ``` + npx @taskless/cli agent auth + ``` + + Follow that recipe with the user. If they do not want an account, + stop here — say the rule cannot run without one, and offer to + reconsider whether a narrower version of the request could be + answered by a static rule instead. That is a new routing decision, + not a fallback you take on their behalf. + +3. **Once they are logged in, re-route.** Run + `npx @taskless/cli info --json` to confirm `loggedIn: true`, then + re-run `taskless agent route` with the original request. The + destination changes now that the gate is open. + +## Important Notes + +- `--dangerously-run-scripts` makes `check` execute runtime rules + without server verification. It exists for local iteration on a rule + you wrote yourself and just read. It is not a way to ship a rule to a + team, and suggesting it to work around a login turns a deliberate gate + into an unreviewed code-execution path on someone else's machine. +- Do not author a `check.ts` and leave it in the repository unmentioned. + A skipped runtime rule reports nothing, which reads exactly like a + passing one. + +## See Also + +- `taskless agent auth` — log in, log out, check status +- `taskless agent route` — re-decide once the login state changes +- `taskless agent check` — see which rules ran and which were skipped diff --git a/packages/cli/src/help/create-sg-rule.txt b/packages/cli/src/help/create-sg-rule.txt new file mode 100644 index 0000000..784c68f --- /dev/null +++ b/packages/cli/src/help/create-sg-rule.txt @@ -0,0 +1,113 @@ +# Topic: create-sg-rule (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `create-sg-rule`. It helps you write an ast-grep rule: a check +over the structure of a single source file, authored on this machine. +If that is not the kind of check you need, re-run `taskless agent route` +and follow its decision rather than adapting this recipe. + +## Goal +Produce a verified ast-grep rule and its test file, locally, without +contacting the Taskless service. You derive the rule yourself, write it +in the canonical on-disk shape, and validate it with `verify-rule` in a +feedback loop. The files you produce match exactly what the service +writes, so `check`, `improve-rule`, and `verify-rule` treat them +identically. + +## Preconditions +- `.taskless/` directory exists. +- The agent can read/write files and run shell commands. +- No auth required. + +## Artifacts + +A rule is complete when both of these exist: + +1. The rule at `.taskless/sg/rules/.yml`. +2. Its tests at `.taskless/sg/rule-tests/-YYYYMMDD-test.yml`. + +## Steps + +1. **Learn the ast-grep rule format.** Consult the ast-grep rule + reference at https://ast-grep.github.io/guide/rule-config.html for + valid fields and operators (`pattern`, `kind`, `regex`, + `any`/`all`/`has`/`inside`/`not`) and meta-variable syntax. This + recipe does not embed the schema — read it from upstream rather than + writing a rule from memory. + +2. **Gather and confirm the pattern.** Make sure you have concrete + success cases (code that should pass) and failure cases (code that + should be flagged), the target language, and any exceptions. Search + the codebase for real instances and confirm exclusions with the user. + +3. **Check for an existing rule that already covers this.** Scan + `.taskless/sg/rules/` and read each rule's `message`, `note`, and + `rule` fields. If one overlaps, show the user and ask whether they + would rather improve it — `taskless agent improve-rule --anonymous` + iterates a rule locally. + +4. **Author the rule in the canonical shape.** Write the rule to + `.taskless/sg/rules/.yml` with at minimum: + - `id` — kebab-case, matching the filename (e.g. `no-eval`) + - `language` — the target language + - `severity` — `error`, `warning`, `info`, or `hint` + - `message` — a concise single-line explanation + - `rule` — the ast-grep rule object + + Optional but useful: `note` (multi-line guidance, supports markdown), + `fix` (auto-fix pattern), `ignores` (file patterns to skip). + +5. **Write the tests.** Write + `.taskless/sg/rule-tests/-YYYYMMDD-test.yml` with the matching + `id` field plus `valid` and `invalid` arrays — at least two of each, + drawn from real code where you can. The `id` must match the rule's + `id` so ast-grep test filtering pairs them. These paths and this + shape are the same ones the service writes; do not invent a different + layout. + +6. **Run the verify feedback loop.** Run: + ``` + npx @taskless/cli rule verify --json + ``` + - `success: true` → the rule passes. Go to step 7. + - `success: false` → read the per-layer errors and fix. Repeat up to + 3 times. + + | layer | what a failure means | fix | + |----------------|-----------------------------------------|------------------------------------------------| + | `schema` | YAML doesn't match the ast-grep schema | check field types against the upstream schema | + | `requirements` | a Taskless-required field is missing | add `id`/`language`/`severity`/`message`/`rule`| + | `tests` | a case didn't behave as expected | fix the rule pattern OR the test case | + + A `regex` without an accompanying `kind` fails `requirements` — the + two always travel together. + +7. **On success, report.** Show the rule and test file paths and a + one-line summary of what the rule detects. Suggest + `taskless agent check` to validate against the broader codebase. + +8. **On failure, escalate — with confirmation.** If after the feedback + loop the rule still cannot capture the user's cases: + - Delete the candidate `.taskless/sg/rules/.yml` and its test file + so the repo is not left with a broken rule. + - Tell the user the local rule could not capture the cases, and that + generating via the Taskless service uses a generation and requires + login. + - Only after the user confirms, fetch `taskless agent + create-remote-rule` and follow it. Do not call the service silently. + +## Important Notes + +- Do NOT make any HTTP requests to taskless.io on this path. +- Do NOT write to `.taskless/rule-metadata/` — locally authored rules + have no metadata sidecar; they iterate via file edits. +- The verify loop is the quality gate. A clean failure is a legitimate + reason to escalate, but only with the user's confirmation (step 8). + +## See Also + +- `taskless agent route` — re-decide the destination +- `taskless agent verify-rule` — the validation primitive step 6 calls +- `taskless agent improve-rule` — iterate on a rule that already exists +- `taskless agent create-remote-rule` — generate via the service (login) +- `taskless agent check` — validate the new rule against the codebase diff --git a/packages/cli/src/help/create-vale-rule.txt b/packages/cli/src/help/create-vale-rule.txt new file mode 100644 index 0000000..38e8fc6 --- /dev/null +++ b/packages/cli/src/help/create-vale-rule.txt @@ -0,0 +1,200 @@ +# Topic: create-vale-rule (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `create-vale-rule`. It helps you write a Vale rule: a check over +the words of a document — prose, markup, and the prose parts of code. +If that is not the kind of check you need, re-run `taskless agent route` +and follow its decision rather than adapting this recipe. + +## Goal +Produce a Vale rule that fires on the prose it should flag and stays +quiet on the prose it should not, scoped to the files it is about. + +## Preconditions +- `.taskless/` directory exists (it contains `vale/.vale.ini`). +- The agent can read/write files and run shell commands. +- No auth required. + +## The three artifacts + +A Vale rule is three files, not one. It is **incomplete until all three +exist**, and the failure mode of skipping one is silence rather than an +error — the rule verifies, the check runs, and nothing is ever reported. + +| # | Artifact | Path | +|---|-----------|-------------------------------------------------------------| +| 1 | The style | `.taskless/vale/rules/.yml` | +| 2 | The scope | a section in `.taskless/vale/.vale.ini` enabling `rules.`| +| 3 | Fixtures | `.taskless/vale/rule-tests//pass/` and `.../fail/` | + +## Steps + +1. **Say what the rule reads, then pick an extension point.** Write one + sentence: "this fires when a document contains ___." Vale rules are + built by extending one of its checks, and the sentence tells you + which: + + | If the rule is about… | extends | + |-------------------------------------------------------|------------------| + | words or phrases that should not appear | `existence` | + | preferring one term over another | `substitution` | + | how something is capitalized (headings, product names)| `capitalization` | + | how many times something may appear | `occurrence` | + | a word repeated back to back | `repetition` | + | picking one of two acceptable spellings, consistently | `consistency` | + | "if X appears, Y must also appear" | `conditional` | + | readability or length thresholds | `metric` | + + There are eleven in total. When none of the above fits, read + https://docs.vale.sh/styles for `spelling`, `sequence`, and `script` + before inventing something — Vale has no facility for a rule that + does not extend one of these. + +2. **Write the style file** to `.taskless/vale/rules/.yml`, where + `` is kebab-case and matches the filename. Every rule carries: + + | Field | Required | Notes | + |-----------|----------|---------------------------------------------------------| + | `extends` | yes | one of the extension points above | + | `message` | yes | shown to the user; `%%s` interpolates the match | + | `level` | no | `suggestion` (default), `warning`, or `error` | + | `scope` | no | narrow to part of a document, e.g. `heading`, `paragraph`| + | `link` | no | a URL the reader can follow for the reasoning | + + Then the fields the chosen extension point adds. The three most + common: + + ```yaml + # existence — flag these tokens wherever they appear + extends: existence + message: "Avoid '%%s' — it hides the work from the reader" + level: warning + ignorecase: true + tokens: + - simply + - just + ``` + + ```yaml + # substitution — flag the observed term, name the expected one + extends: substitution + message: "Use '%%s' instead of '%%s'" + level: warning + ignorecase: true + swap: + utilize: use + "in order to": to + ``` + + ```yaml + # capitalization — enforce a case pattern on a part of the document + extends: capitalization + message: "'%%s' should be in sentence case" + level: warning + scope: heading + match: $sentence + exceptions: + - Taskless + - API + ``` + + `match` takes `$sentence`, `$title`, `$lower`, `$upper`, or a literal + string to enforce verbatim (`Taskless`, for a product name). + +3. **Scope the rule by adding a section to `.taskless/vale/.vale.ini`.** + This is the step that is easy to skip and impossible to notice + skipping. + + A scaffolded project ships **no section at all** — only `StylesPath` + and `MinAlertLevel` — because scope is the author's decision and the + scaffold will not guess it. So the first rule in a project also + writes the first section: + + ```ini + StylesPath = . + MinAlertLevel = suggestion + + [*.md] + BasedOnStyles = + rules.no-simply = YES + ``` + + Three things about that block: + - `[*.md]` is a glob over paths. Match it to the files the rule is + actually about — `[*.{md,mdx}]`, `[docs/**/*.md]`. Every file the + walk reaches that matches the glob is subject to every rule + enabled under it. + - `BasedOnStyles =` left empty says explicitly that no bundled style + loads, so the only findings are the rules you enabled. Write it + even though the default is already empty: it is what stops a later + edit from switching a whole style on by accident, and it is the + form `taskless check` uses when it verifies a rule in isolation. + - `rules. = YES` enables the rule. `rules` is the style name: + `StylesPath = .` makes the `rules/` directory a style, so a file at + `vale/rules/no-simply.yml` is the check `rules.no-simply`. + + If a section already covers the right files, add the rule to it + rather than opening a second one — a later section matching the same + files wins outright. + + **A rule enabled outside a section does nothing.** Put + `rules.no-simply = YES` above the first `[…]` line and Vale does not + error: it prints `W101 'rules.no-simply' isn't a core option` to + stderr, exits zero, and reports no findings. The CLI surfaces that + diagnostic as a notice — if you see it, the assignment is in the + wrong place, not the rule. + +4. **Write the fixtures.** Two directories, both flat — Vale lints the + rule's whole tree, so a nested subdirectory is linted but never + checked, and the CLI rejects it rather than skipping it: + + ``` + .taskless/vale/rule-tests//pass/ok.md # rule must stay quiet + .taskless/vale/rule-tests//fail/bad.md # rule must fire + ``` + + Both buckets are required. A rule with only `fail/` fixtures has + never been shown to stay quiet, which is the half that catches an + over-broad pattern. + + Give the fixtures an extension your section's glob matches. A `.txt` + fixture under a `[*.md]` section is never linted, so the `fail/` + document silently passes. + +5. **Check that it fires, and that it doesn't.** Run the rule against + each bucket: + + ``` + npx @taskless/cli check .taskless/vale/rule-tests//fail --json + npx @taskless/cli check .taskless/vale/rule-tests//pass --json + ``` + + The `fail/` run must report your rule; the `pass/` run must report + nothing. If `fail/` reports nothing, work down the list before + touching the pattern — the cause is usually further up: + - Is the assignment inside a section? (step 3) + - Does the section's glob match the fixture's extension? (step 4) + - Does `rules.` match the filename under `vale/rules/`? + - Only then: does the pattern actually match the text? + +6. **Report.** Show the three paths you wrote, a one-line summary of + what the rule flags, and which files the section scopes it to — the + scope is a decision the user should see, not one buried in a config. + Suggest `taskless agent check` to run it over the whole project. + +## Important Notes + +- Vale reads one document at a time and has no cross-document view. A + rule about consistency *between* documents cannot be written here. +- Prose inside code is still prose: comments and docstrings are Vale's + subject, and a rule about them belongs in a section whose glob covers + the source files. +- Do NOT add a `[*]` section to widen a rule that isn't firing. It + applies every enabled rule to every file the walk reaches, which turns + one rule's scoping bug into every rule's false positives. + +## See Also + +- `taskless agent route` — re-decide the destination +- `taskless agent check` — run every engine over the repo +- `taskless agent create-sg-rule` — author a rule over code structure diff --git a/packages/cli/src/help/rule-delete.txt b/packages/cli/src/help/delete-rule.txt similarity index 100% rename from packages/cli/src/help/rule-delete.txt rename to packages/cli/src/help/delete-rule.txt diff --git a/packages/cli/src/help/engine-selection.txt b/packages/cli/src/help/engine-selection.txt deleted file mode 100644 index 15ffed6..0000000 --- a/packages/cli/src/help/engine-selection.txt +++ /dev/null @@ -1,113 +0,0 @@ -# Topic: engine-selection (CLI v%(CLI_VERSION)s / topic v1) - -## Goal -Decide WHICH ENGINE enforces a rule: `sg` (ast-grep), `vale`, or -`runtime`. This is a question about the shape of the evidence the rule -needs, not about where the rule gets authored and not about how much -the rule is trusted. - -## Preconditions -- A rule intent stated in words ("no `eval`", "don't say 'simply'", - "every exported symbol must appear in the changelog"). -- No auth required. Engine choice is a property of the rule, not of the - user's session. - -## The three engines - -**`sg` — one file's syntax tree.** -Code structure, decided by parsing a single file: a call, an import, a -JSX attribute, a type annotation. Relational correlation counts as -in-file — "a `useEffect` whose dependency array omits a value used in -its body" is still one file's tree, and `sg` handles it. If the rule -can be settled by looking at exactly one file's structure, it is `sg`. - -**`vale` — prose and markup.** -Natural language in documents and in the prose parts of code: docs, -README, comments, commit-message bodies. Word choice, phrasing, -terminology, capitalization, banned or preferred vocabulary. Vale reads -a document, not a syntax tree. - -**`runtime` — everything that needs more than one file, or needs the -world.** -Cross-file and graph questions ("this exported symbol is unused across -the repo"), metadata not in any file's text (git history, ownership, -build outputs), and normalization a parser cannot express (resolving -aliases, following a config chain). If answering the rule requires -holding two files at once, or looking outside the files, it is -`runtime`. - -## Steps - -1. **State the evidence the rule needs, before naming an engine.** - Write one sentence: "to decide this, you must look at ___." Do this - first, every time. Naming an engine and then justifying it is how - prose-about-code ends up in `sg` and cross-file questions end up in - `vale`. - -2. **Match the evidence to an engine.** - - one file's structure → `sg` - - a document's words → `vale` - - more than one file, or anything outside the files → `runtime` - -3. **Check the boundary cases below.** Most wrong answers are one of - the three. - -4. **If it is still ambiguous, choose an engine you know is - available.** State that as the reason. Both `sg` and `vale` ship as - platform binaries with the CLI, so both are normally present; on an - unsupported architecture, or where an install was blocked, the - fallback may find nothing. Server-side the same conclusion follows - from a different cause: `sg` is the only ungated route. Prefer the - engine whose availability you can actually assert in the situation - you are in, rather than defaulting to a name. - -## Boundary cases - -**Prose about code is still prose.** -"Comments must not say 'obviously'" is `vale` — the evidence is the -words. "Every exported function has a doc comment" is `sg` — the -evidence is whether a node exists above a declaration. Ask what you -would have to read to decide, not what the subject matter is. - -**Vale sees one document at a time.** -Vale has no cross-document view. "This term is spelled consistently -ACROSS the docs directory" is `runtime`, not `vale`, even though it is -entirely about prose. Per-document prose is `vale`; prose consistency -between documents is a graph question. - -**Engine is not trust tier.** -`sg` and `vale` are both static-tier: inert data, always run, no login, -no reconcile, no signing. Only `runtime` involves verification, because -only `runtime` executes code. "Static vs runtime" is a different axis -from "which engine", and conflating them is what makes a prose rule -look like it needs login. - -## Worked examples - -| Rule intent | Evidence needed | Engine | -| -------------------------------------------------------- | ---------------------------------------- | --------- | -| No `eval(...)` anywhere | one file's call expressions | `sg` | -| `useEffect` deps must include what the body reads | one file's tree, correlated within it | `sg` | -| Don't write "simply" or "just" in docs | a document's words | `vale` | -| Comments must not say "obviously" | a document's words (comments are prose) | `vale` | -| Headings use sentence case | a document's markup | `vale` | -| Exported symbols must be used somewhere in the repo | every file, correlated | `runtime` | -| Product name spelled the same across all docs | many documents, compared | `runtime` | -| Files changed in the last release need a changelog entry | git metadata, not file contents | `runtime` | -| Imports must resolve through the tsconfig path aliases | config chain resolution, outside the file| `runtime` | - -## Important Notes - -- This topic decides the engine and nothing else. Where a rule is - authored — in a linter the repo already uses, locally, or via the - service — is `route`, and it is a separate decision made earlier. - Locally the two compose: route first, then engine. -- A rule that seems to need two engines is usually two rules. Split it - and let each half pick its own engine, rather than forcing one engine - to cover evidence it cannot see. - -## See Also - -- `taskless help route` — decide WHERE to author a rule (a different question) -- `taskless help static` — author a local ast-grep rule (no login) -- `taskless help check` — run every engine over the repo diff --git a/packages/cli/src/help/rule-improve.anonymous.txt b/packages/cli/src/help/improve-rule.anonymous.txt similarity index 100% rename from packages/cli/src/help/rule-improve.anonymous.txt rename to packages/cli/src/help/improve-rule.anonymous.txt diff --git a/packages/cli/src/help/rule-improve.txt b/packages/cli/src/help/improve-rule.txt similarity index 100% rename from packages/cli/src/help/rule-improve.txt rename to packages/cli/src/help/improve-rule.txt diff --git a/packages/cli/src/help/remote.txt b/packages/cli/src/help/remote.txt deleted file mode 100644 index 4188ad2..0000000 --- a/packages/cli/src/help/remote.txt +++ /dev/null @@ -1,60 +0,0 @@ -# Topic: remote (CLI v%(CLI_VERSION)s / topic v1) - -## Goal -Generate a rule using the Taskless service. Your job on this path is to -gather the inputs and hand off to `rule create`; the service generates -the rule and writes the standard rule files. This path consumes a -generation and requires login. Reach it when a rule is not reasonably -solvable locally, or when a believed-local attempt failed and the user -confirmed. - -## Preconditions -- `.taskless/` directory exists. -- The user is logged in (this path requires auth). -- The request has been routed here (see `taskless help route`), not - reached by skipping the local-first decision. - -## Steps - -1. **Confirm this is the right path.** You should be here because the - request is not reasonably solvable locally, OR a local `static` - attempt failed and the user confirmed spending a generation. If the - user has not confirmed an escalation from a failed local attempt, get - that confirmation first — the service costs a generation and login. - -2. **Confirm auth.** Run: - ``` - npx @taskless/cli info --json - ``` - Check `loggedIn`. If false, fetch `taskless help auth` and follow the - login recipe before continuing. - -3. **Gather the request.** Collect the rule description plus concrete - success and failure cases. If you arrived here from `static`, reuse - the cases you already gathered. The service uses these to generate the - rule. - -4. **Delegate to the generation backend.** Fetch `taskless help rule - create` and follow it. That recipe builds the request payload and - invokes `rule create`, which submits to the service and writes the - result. Do not re-implement the submission here. - -5. **Report results.** The service writes the generated rule to the same - on-disk paths and shape as a locally authored rule, so `check`, - `improve`, and `verify` treat it identically. Show the user the file - paths and suggest `taskless help check` to validate. - -## Important Notes - -- Do NOT pre-build the rule yourself on this path — submit the request - and let the service generate it. -- The service owns rule-type selection. Today it generates ast-grep - rules written under `.taskless/sg/rules/`, the same shape the local - `static` path produces. - -## See Also - -- `taskless help route` — the local-first routing decision -- `taskless help rule create` — the generation backend this path uses -- `taskless help auth` — log in before generating -- `taskless help check` — validate the generated rule diff --git a/packages/cli/src/help/route.txt b/packages/cli/src/help/route.txt index b39ac71..aaf7fca 100644 --- a/packages/cli/src/help/route.txt +++ b/packages/cli/src/help/route.txt @@ -1,23 +1,23 @@ # Topic: route (CLI v%(CLI_VERSION)s / topic v1) ## Goal -Decide where a rule-authoring request should be built: in a linter the -repo ALREADY uses (`existing`), as a local ast-grep rule on this machine -(`static`), or by the Taskless service (`remote`). This is the front -door for "author/write/create a rule" requests. You stay local whenever -you can reasonably build the rule on-device; you only send the user to -the login-gated service when you are not reasonably confident the rule -is locally solvable — or when a believed-local attempt has genuinely -failed and the user confirms. +Turn "write me a rule that…" into one command to run. This is the front +door for every rule-authoring request, and it makes one decision — which +of five recipes authors this rule — from one reading of the evidence. + +That single decision covers both halves that used to be asked +separately: whether the rule can be built here, and which engine can +express it. They are answered from the same signals, so they are +answered together. ## Preconditions - A working repository the agent can read. -- No auth required to route. (The `remote` destination requires login; - the `existing` and `static` destinations do not.) +- No auth required to route. Login state is an *input* to the decision, + not a requirement of making it. ## Steps -1. **Scan the repo.** Run: +1. **Read the repository.** Run: ``` npx @taskless/cli detect --json ``` @@ -36,62 +36,151 @@ failed and the user confirms. } ``` -2. **Write your reasoning BEFORE naming a destination.** Do not pick a - route first and justify it after. Write a short rationale covering: - - what the `detect` signals show (linters, languages, repo rule styles); - - whether an existing linter plausibly already covers this request; - - whether the pattern is expressible as a simple static ast-grep rule; - - your resulting confidence that the request is locally solvable. - The destination you choose in step 3 MUST follow from this rationale. - -3. **Choose the destination as a conclusion of the rationale:** - - - **Both an existing linter AND a local static rule fit** → do NOT - silently pick one. Present both to the user with trade-offs and let - them choose. Note that `remote` consumes a Taskless generation and - requires login, so it is for what cannot be solved locally — not a - default. - - **An existing linter clearly fits** (the repo uses it and it can - express this) → fetch `taskless help existing`. - - **You are reasonably confident it is a simple static ast-grep - pattern** → fetch `taskless help static`. Reasonable confidence is - enough here; you do not need certainty, because step 4 backstops a - wrong-but-reasonable bet. - - **You reasonably believe it CANNOT be solved locally** (a positive - judgment — e.g. it needs cross-file or semantic checks ast-grep can't - express) → fetch `taskless help remote`. Do not manufacture a - deliberate local failure to reach this; route here directly. - - These are three distinct states. If you genuinely cannot tell whether - the request is local or remote — mere uncertainty, not a belief that it - needs the service — ASK the user a clarifying question. Uncertainty is a - reason to ask, never a reason to spend a generation on `remote`. - -4. **Failure fallback (try-verify-escalate).** If you committed to a - local `static` rule on reasonable confidence and it then FAILS - verification against the user's success/failure cases: - - Tell the user the local rule could not capture the cases. - - State that generating via the Taskless service uses a generation - and requires login. - - Call the service only after the user confirms. On yes, fetch - `taskless help remote`. - Never silently fall through from a failed local attempt to a service - call — the confirmation step is required. +2. **Read the login state.** Run: + ``` + npx @taskless/cli info --json + ``` + and note `loggedIn`. Do this now, not later: it changes which + destinations exist, so classifying first means classifying against a + set that may be wrong. + + Reading it is not the same as asking about it. Do **not** open by + offering service generation — at this point neither you nor the user + knows whether this is a two-line pattern or something local authoring + cannot express, so the question costs a turn and cannot be answered + well. + +3. **State the evidence before you name a destination.** Write one + sentence: *"to decide this, you must look at ___."* Then a short + rationale covering what `detect` showed, whether a linter the repo + already runs could express this, and what the rule needs to read. + + Do this first, every time. Naming a destination and justifying it + afterwards is how prose-about-code ends up in `sg` and cross-file + questions end up in `vale`. + +4. **Match the evidence to a destination.** The comparison is made here + and only here — the destination recipes describe their own scope and + deliberately do not restate this table. + + | The rule is decided by… | Destination | Login | + |-----------------------------------------------------------------|----------------------|-------| + | a tool the repo already runs, in that tool's dialect | `create-legacy-rule` | no | + | **one file's syntax tree** — a call, an import, a JSX attribute, a type annotation | `create-sg-rule` | no | + | **a document's words** — docs, README, comments, commit bodies | `create-vale-rule` | no | + | **more than one file, or something outside the files** — the repo graph, git metadata, build output, a resolved config chain | `create-runtime-rule` / `create-remote-rule` | see step 5 | + + Sharpening the three engine rows, because most wrong answers are one + of these: + + - **Relational correlation inside one file is still `sg`.** "A + `useEffect` whose dependency array omits a value used in its body" + is one file's tree. If exactly one file settles it, it is `sg`. + - **Prose about code is still prose.** "Comments must not say + 'obviously'" is `vale` — the evidence is the words. "Every exported + function has a doc comment" is `sg` — the evidence is whether a node + exists above a declaration. Ask what you would have to *read* to + decide, not what the subject matter is. + - **Vale sees one document at a time.** "This term is spelled + consistently ACROSS the docs directory" is a graph question, so it + is runtime — even though it is entirely about prose. + - **Trust tier is not a destination.** `sg` and `vale` are both + static-tier: inert data, always run, no login, no reconcile, no + signing. Only runtime executes code. "Static vs runtime" is a + different axis from "which engine", and conflating them is what + makes a prose rule look like it needs an account. + + Worked examples: + + | Rule intent | Evidence needed | Destination | + |-----------------------------------------------------------|-------------------------------------------|----------------------| + | No `eval(...)` anywhere | one file's call expressions | `create-sg-rule` | + | `useEffect` deps must include what the body reads | one file's tree, correlated within it | `create-sg-rule` | + | Don't write "simply" or "just" in docs | a document's words | `create-vale-rule` | + | Comments must not say "obviously" | a document's words (comments are prose) | `create-vale-rule` | + | Headings use sentence case | a document's markup | `create-vale-rule` | + | Exported symbols must be used somewhere in the repo | every file, correlated | runtime | + | Product name spelled the same across all docs | many documents, compared | runtime | + | Files changed in the last release need a changelog entry | git metadata, not file contents | runtime | + | Imports must resolve through the tsconfig path aliases | config chain resolution, outside the file | runtime | + +5. **Split the runtime row on login state.** Runtime rules execute code, + so they run only against a server-verified signature: + - **logged in** → `create-remote-rule`. Go there directly; no recipe + forwards to another. + - **logged out** → `create-runtime-rule`, which explains the gate. + +6. **Offer the service only where it is a genuine choice.** When the + rule is expressible locally **and** the user is logged in, present + both and let them pick: a local rule you write and verify now, or a + generation that costs against their plan. Say which you would choose + and why. + + Not logged in, or not locally expressible, are not choices — do not + pose them as one. + +7. **Name the command.** Finish by telling the user, or running, the + exact fetch for the destination you chose: + ``` + npx @taskless/cli agent create-vale-rule + ``` + A destination that is not a runnable command is a category, and a + category is not an answer. + +## When two destinations both fit + +- **A linter the repo already runs AND a local Taskless rule both fit** → + do not silently pick. Present both with trade-offs and let the user + choose. +- **A rule that seems to need two engines is usually two rules.** Split + it and route each half, rather than forcing one engine to cover + evidence it cannot see. +- **You genuinely cannot tell** → ask the user a clarifying question. + Uncertainty is a reason to ask; it is never a reason to spend a + generation. + +## Failure fallback (try-verify-escalate) + +If you committed to a local rule on reasonable confidence and it then +fails against the user's cases: + +- Tell the user the local rule could not capture them. +- State that generating via the service uses a generation and requires + login. +- Call the service only after the user confirms. On yes, fetch + `taskless agent create-remote-rule`. + +Never fall through silently from a failed local attempt to a service +call. A developer who watches a local attempt fail reads it as a +Taskless failure, so only attempt local when you reasonably believe it +will work. ## Important Notes -- Reason first, route second. Do not name a destination before the - rationale is written. -- Stay local when you reasonably can. `remote` is the path for what - cannot be solved on-device, and it costs a generation plus login. -- A developer who watches a local attempt fail reads it as a Taskless - failure. Only attempt local when you reasonably believe it will work; - otherwise route `remote` upfront. +- **Reason first, route second.** Do not name a destination before the + rationale in step 3 is written. +- **What the code actually looks like outranks how the request was + phrased.** Where you can see the code or the diff, weigh the concrete + syntactic form present in the repository above the wording. The same + sentence routes to `create-sg-rule` when the repository contains a + form that correlates statically, and to runtime when the equivalent + check would have to normalize a captured value against a declaration + elsewhere. +- **When it stays ambiguous, choose an engine whose availability you can + assert, and say that is why.** There is no fixed fallback. `sg` and + `vale` both ship as platform binaries, so either can be the missing one + on an unsupported architecture or where an install was blocked; + server-side the constraint is different again. A named default is + wrong in whichever situation it failed to anticipate. +- **Stay local when you reasonably can.** Reasonable confidence is + enough to commit to a local rule — the failure fallback backstops a + wrong-but-reasonable bet. ## See Also -- `taskless help existing` — author in a linter the repo already uses -- `taskless help static` — author a local ast-grep rule (no login) -- `taskless help remote` — generate via the Taskless service (login) -- `taskless help engine-selection` — decide WHICH engine enforces the rule - (the next decision once the destination is settled) +- `taskless agent create-legacy-rule` — author in a linter the repo already uses +- `taskless agent create-sg-rule` — author a local ast-grep rule (no login) +- `taskless agent create-vale-rule` — author a local Vale rule (no login) +- `taskless agent create-runtime-rule` — the runtime tier, logged out +- `taskless agent create-remote-rule` — generate via the service (login) +- `taskless agent check` — run every engine over the repo diff --git a/packages/cli/src/help/rule-create.anonymous.txt b/packages/cli/src/help/rule-create.anonymous.txt deleted file mode 100644 index 74c88de..0000000 --- a/packages/cli/src/help/rule-create.anonymous.txt +++ /dev/null @@ -1,112 +0,0 @@ -# Topic: rule create (anonymous) (CLI v%(CLI_VERSION)s / topic v1) - -## Goal -Create a new ast-grep rule **locally** without contacting the Taskless -API. You (the agent) derive the rule yourself using the ast-grep -schema as a guide, write the rule and test files, then validate them -using `rule verify` in a feedback loop. - -## Preconditions -- `.taskless/` directory exists. -- The agent can read/write files and run shell commands. -- No auth required. - -## Steps - -1. **Learn the ast-grep rule format.** Before writing a rule, consult - the ast-grep rule reference at https://ast-grep.github.io/guide/rule-config.html - for valid fields, operators (`pattern`, `kind`, `regex`, - `any`/`all`/`has`/`inside`/`not`), and meta-variable syntax. The - Taskless-required fields you'll add on top are listed in step 5. - -2. **Gather the rule description.** Even if the user already provided - one, ask clarifying questions: - - What specific code pattern should be flagged? (concrete examples) - - What language is it in? - - Are there exceptions or edge cases where the pattern is OK? - - Can they show valid and invalid examples? - -3. **Check for similar existing rules.** Scan `.taskless/sg/rules/`. If - anything overlaps, point it out and ask whether the user wants to - improve an existing rule via `taskless help rule improve --anonymous`. - -4. **Search the codebase for real instances.** Show the user what you - found and confirm any exclusions. - -5. **Derive the rule.** Write a YAML file at `.taskless/sg/rules/.yml` - with at minimum: - - `id`: kebab-case identifier (e.g. `no-eval`, `prefer-const`) - - `language`: target language - - `severity`: `error`, `warning`, `info`, or `hint` - - `message`: concise single-line explanation - - `rule`: the ast-grep rule object - - Optional but useful: `note` (multi-line guidance, supports markdown), - `fix` (auto-fix pattern), `ignores` (file patterns to skip). - -6. **Write test cases.** Create `.taskless/sg/rule-tests/-YYYYMMDD-test.yml` - with `id`, `valid: [...]`, and `invalid: [...]` arrays. Include at - least 2 valid and 2 invalid cases. Use real patterns from the - codebase where possible. - -7. **Run the verify feedback loop.** Run: - ``` - npx @taskless/cli rule verify --json - ``` - - If `success: true`: the rule passes. Report success to the user. - - If `success: false`: read the per-layer errors (`schema`, - `requirements`, `tests`). Fix the rule or tests. Re-run verify. - Repeat up to 3 times. If still failing after 3 iterations, report - to the user with the latest errors. - - Common fixes: - - Schema errors → check field types against the ast-grep schema. - - Missing required fields → add what the requirements list demands. - - Regex without kind → add a `kind` field alongside any `regex`. - - Test failures → adjust pattern or test cases so valid cases pass - and invalid cases trigger. - -8. **Report results.** Show the rule file path, test file path, and a - one-line summary of what the rule detects. Suggest fetching - `taskless help check` to validate against the broader codebase. - -## Important Notes - -- Do NOT write files to `.taskless/rule-metadata/`. Anonymous rules - have no metadata sidecar (the API path uses metadata for - iteration; anonymous rules iterate directly via file edits). -- Do NOT make any HTTP requests to taskless.io. -- The verify feedback loop is the quality gate — always run it before - reporting success. - -## ast-grep schema - -This recipe does not embed the full ast-grep schema. Read it from the -upstream docs at https://ast-grep.github.io/guide/rule-config.html -before authoring a rule. The Taskless-specific required fields layered -on top of the ast-grep rule are listed in step 5. - -## Errors - -`rule verify --json` has two output shapes depending on whether the -`` argument is supplied: - -- When `` is missing, the command emits the standardized - `{ ok:false, code:"INVALID_INPUT", message }` envelope on stdout - and exits non-zero. -- When `` is supplied, the command emits the layered result - `{ success, schema, requirements, tests, ... }` on stdout (success - or failure exit code depending on the layers). - -| where | code / shape | meaning | fix | -|---------------------|-----------------|------------------------------------------|--------------------------------------------| -| envelope (no ``)| `INVALID_INPUT` | Rule ID missing | Pass a valid rule ID positionally | -| layered: schema | (schema layer) | YAML didn't match ast-grep schema | Fix the rule structure | -| layered: req | (req layer) | Missing Taskless-required field | Add `id`/`language`/`severity`/etc. | -| layered: tests | (tests layer) | A test case didn't behave as expected | Fix the rule pattern OR the test case | - -## See Also - -- `taskless help rule create` — API-backed flow (auth required) -- `taskless help rule improve --anonymous` — iterate locally -- `taskless help check` — validate the new rule against the codebase diff --git a/packages/cli/src/help/rule-create.txt b/packages/cli/src/help/rule-create.txt deleted file mode 100644 index 6cbc60e..0000000 --- a/packages/cli/src/help/rule-create.txt +++ /dev/null @@ -1,101 +0,0 @@ -# Topic: rule create (CLI v%(CLI_VERSION)s / topic v1) - -## Goal -Generate a new ast-grep rule from a description and write the rule and -its tests to the user's `.taskless/` directory. The CLI calls the -Taskless API to do the generation; the agent's job is to enrich the -user's description before submitting and to report the result. - -If the user wants the local-only flow (no API call), fetch -`taskless help rule create --anonymous` instead. - -## Preconditions -- User is logged in (`taskless info --json` reports `loggedIn: true`). - If not, fetch `taskless help auth` first. -- Repository has a GitHub origin remote. -- `.taskless/` directory exists (otherwise the user has not run - `npx @taskless/cli` to install Taskless). - -## Steps - -1. **Confirm auth.** Run `npx @taskless/cli info --json` and check the - `loggedIn` field. If false, fetch `taskless help auth` and follow the - login recipe before continuing. If the user explicitly wants - anonymous mode, fetch `taskless help rule create --anonymous` instead. - -2. **Gather the rule description.** Even if the user already provided - one, ask clarifying questions: - - What specific code pattern should be flagged? (concrete examples) - - In what language? - - Are there contexts where the pattern is acceptable? - -3. **Check for similar existing rules.** Scan `.taskless/sg/rules/` for - rule files. Read each rule's `message`, `note`, and `rule` fields. - If any overlap with the user's request, show the user and ask: - "It looks like you already have a rule that covers something - similar. Would you like to improve the existing rule instead?" - If yes, fetch `taskless help rule improve` instead. - -4. **Enrich the request.** - - Search the codebase for real instances of the pattern. Show the - user what you found. - - Ask for additional success and failure cases. - - Read `.gitignore`, ESLint configs, and `tsconfig.json` `exclude` - for default exclusion patterns. Present them as defaults. - - Ask about additional exclusions (e.g. `.d.ts` files, tests). - - Infer the primary language from the codebase. Confirm with the - user. Include it in the `prompt` field. - -5. **Confirm the enriched request.** Before submitting, summarize: - - The full prompt (including language and exclusion notes) - - The success case(s) - - The failure case(s) - -6. **Write the JSON payload.** Build a JSON object matching the input - schema below and write it to `.taskless/.tmp-rule-request.json`. - -7. **Invoke the CLI.** Run: - ``` - npx @taskless/cli rule create --from .taskless/.tmp-rule-request.json --json - ``` - This may take 30–60 seconds while the API generates the rule. - -8. **Clean up.** Delete `.taskless/.tmp-rule-request.json` regardless - of success or failure. - -9. **Report results.** The CLI writes the generated rule to - `.taskless/sg/rules/.yml`, tests to - `.taskless/sg/rule-tests/-YYYYMMDD-test.yml` (timestamped per - generation), and metadata to `.taskless/rule-metadata/.yml`. - Show the user the file paths. Suggest fetching `taskless help check` - to validate. - -## Input schema - -The `--from` JSON file conforms to: - -```json -%(INPUT_SCHEMA)s -``` - -Each example in `successCases` and `failureCases` is a separate -string. Multi-line code goes in a single string with literal newlines. - -## Errors - -When `--json` is set, failures emit `{ ok: false, code, message }`: - -| code | meaning | fix | -|--------------------------|------------------------------------|---------------------------------------------| -| `AUTH_REQUIRED` | not logged in | fetch `taskless help auth` | -| `NO_GITHUB_REMOTE` | no GitHub origin remote | tell the user; we cannot proceed | -| `INVALID_INPUT` | `--from` JSON failed validation | re-read the input schema, fix, retry | -| `NETWORK_ERROR` | API submit/poll failed | report and suggest retry | -| `RULE_GENERATION_FAILED` | API returned a generation failure | report the message; suggest enriching prompt | -| `RULE_UNSUPPORTED` | plan lacks this generation type | tell the user to enable it; do not retry | - -## See Also - -- `taskless help rule create --anonymous` — local-only flow (no API) -- `taskless help rule improve` — iterate on an existing rule -- `taskless help check` — validate the new rule against the codebase diff --git a/packages/cli/src/help/static.txt b/packages/cli/src/help/static.txt deleted file mode 100644 index a67f6b2..0000000 --- a/packages/cli/src/help/static.txt +++ /dev/null @@ -1,76 +0,0 @@ -# Topic: static (CLI v%(CLI_VERSION)s / topic v1) - -## Goal -Author a Taskless ast-grep rule **locally**, on this machine, without -contacting the Taskless service. You derive the rule yourself, write it -in the canonical on-disk shape, and validate it with `rule verify` in a -feedback loop. The files you produce match exactly what the service -writes, so `check`, `improve`, and `verify` treat them identically. - -## Preconditions -- `.taskless/` directory exists. -- The agent can read/write files and run shell commands. -- No auth required. - -## Steps - -1. **Learn the ast-grep rule format.** Consult the ast-grep rule - reference at https://ast-grep.github.io/guide/rule-config.html for - valid fields and operators (`pattern`, `kind`, `regex`, - `any`/`all`/`has`/`inside`/`not`) and meta-variable syntax. - -2. **Gather and confirm the pattern.** Make sure you have concrete - success cases (code that should pass) and failure cases (code that - should be flagged), the target language, and any exceptions. Search - the codebase for real instances and confirm exclusions with the user. - -3. **Author the rule in the canonical shape.** Write the rule to - `.taskless/sg/rules/.yml` with at minimum `id` (kebab-case), - `language`, `severity` (`error`/`warning`/`info`/`hint`), `message`, - and the `rule` object. Write tests to - `.taskless/sg/rule-tests/-YYYYMMDD-test.yml` with the matching `id` - field plus `valid` and `invalid` arrays (at least two of each). The - `id` must match the rule's `id` so ast-grep test filtering pairs them. - These paths and shape are the same ones the service writes — do not - invent a different layout. - -4. **Run the verify feedback loop.** Run: - ``` - npx @taskless/cli rule verify --json - ``` - - `success: true` → the rule passes. Go to step 5. - - `success: false` → read the per-layer errors (`schema`, - `requirements`, `tests`), fix the rule or tests, and re-run. Repeat - up to 3 times. - -5. **On success, report.** Show the rule and test file paths and a - one-line summary of what the rule detects. Suggest `taskless help - check` to validate against the broader codebase. - -6. **On failure, escalate via the route fallback — with confirmation.** - If after the feedback loop the rule still cannot capture the user's - cases, this is the try-verify-escalate fallback: - - Delete the candidate `.taskless/sg/rules/.yml` and its test file so - the repo is not left with a broken rule (guaranteed cleanup of the - abandoned candidate). - - Tell the user the local rule could not capture the cases, and that - generating via the Taskless service uses a generation and requires - login. - - Only after the user confirms, fetch `taskless help remote` and - follow it. Do not call the service silently. - -## Important Notes - -- Do NOT make any HTTP requests to taskless.io on this path. -- Do NOT write to `.taskless/rule-metadata/` — local rules have no - metadata sidecar; they iterate via file edits. -- The verify loop is the quality gate. A clean failure is a legitimate - reason to escalate, but only with the user's confirmation (step 6). - -## See Also - -- `taskless help route` — re-decide the destination -- `taskless help remote` — generate via the Taskless service (login) -- `taskless help engine-selection` — confirm `sg` is the right engine for - this rule, rather than `vale` or `runtime` -- `taskless help check` — validate the new rule against the codebase diff --git a/packages/cli/src/help/rule-verify.txt b/packages/cli/src/help/verify-rule.txt similarity index 100% rename from packages/cli/src/help/rule-verify.txt rename to packages/cli/src/help/verify-rule.txt diff --git a/packages/cli/src/prompts/index.ts b/packages/cli/src/prompts/index.ts index f7e1017..7e58548 100644 --- a/packages/cli/src/prompts/index.ts +++ b/packages/cli/src/prompts/index.ts @@ -25,18 +25,24 @@ import { getRecipe, type RecipeOptions } from "./recipes.js"; * {@link INTERNAL_TOPICS} accounts for every canonical recipe on disk. * * The list starts at what a consumer has actually asked for and grows on - * demand. `static` is the canonical on-disk rule shape, the one topic the - * generator's decision router can use server-side. + * demand. It is the authoring recipe for each engine a rule can be routed to, + * so a consumer that can decide a rule belongs to an engine can also reach the + * procedure for writing one. Exporting a chooser without its destinations + * reproduces, for the platform generator, the dead end this surface exists to + * remove. * - * `engine-selection` is exported for the same consumer and the same reason - * (D7 of `add-vale-rule-engine`). Choosing an engine is a decision the service - * already makes — today as a binary `static | runtime` classifier that predates - * Vale, and is therefore now wrong. Exporting the topic is what lets the - * generator render the same guidance `taskless help` serves instead of keeping - * a second copy free to drift, which is the divergence this module exists to - * prevent. + * `engine-selection` used to be exported alongside them. It no longer exists: + * the criterion it carried now lives in `route`, stated once. `route` is not + * exported yet because it still contains local mechanics (`taskless detect`, + * on-device authoring) a Worker cannot run; until it is, a consumer gets each + * destination's own scope from these three and adjudicates genuinely ambiguous + * calls itself. */ -export const TOPICS = ["static", "engine-selection"] as const; +export const TOPICS = [ + "create-sg-rule", + "create-vale-rule", + "create-runtime-rule", +] as const; /** * Recipes deliberately withheld from the export, recorded so they stay visible @@ -46,29 +52,28 @@ export const TOPICS = ["static", "engine-selection"] as const; * subcommand on a developer's machine. There is no caller for them outside * the CLI that hosts those commands. * - Authoring recipes are unreachable server-side: `route` picks an authoring - * destination before the service is involved, `remote` states the boundary - * from the client's side, `detect` documents a CLI subprocess a Worker - * cannot spawn, `existing` targets a local toolchain, and `rule-meta` reads - * a `rule improve` sidecar file. + * destination before the service is involved, `create-remote-rule` states + * the boundary from the client's side, `detect` documents a CLI subprocess a + * Worker cannot spawn, `create-legacy-rule` targets a local toolchain, and + * `rule-meta` reads an `improve-rule` sidecar file. */ export const INTERNAL_TOPICS = [ "auth", "check", "ci", + "create-legacy-rule", + "create-remote-rule", + "delete-rule", "detect", - "existing", + "improve-rule", "info", "init", "onboard", - "remote", "route", "rule", - "rule-create", - "rule-delete", - "rule-improve", "rule-meta", - "rule-verify", "update", + "verify-rule", ] as const; /** A topic name the package exports. Unknown names fail to type-check. */ diff --git a/packages/cli/src/prompts/recipes.ts b/packages/cli/src/prompts/recipes.ts index 423e841..0815c54 100644 --- a/packages/cli/src/prompts/recipes.ts +++ b/packages/cli/src/prompts/recipes.ts @@ -58,8 +58,8 @@ export function canonicalRecipeTopics(): string[] { // placeholder, the renderer substitutes the JSON Schema rendered from // this Zod source. const TOPIC_INPUT_SCHEMAS: Record = { - "rule-create": ruleCreateInputSchema, - "rule-improve": ruleImproveInputSchema, + "create-remote-rule": ruleCreateInputSchema, + "improve-rule": ruleImproveInputSchema, }; /** Agent-fill marker used when the caller does not supply a real value. */ diff --git a/packages/cli/src/rules/dispatch.ts b/packages/cli/src/rules/dispatch.ts index e51d106..ec6eb6c 100644 --- a/packages/cli/src/rules/dispatch.ts +++ b/packages/cli/src/rules/dispatch.ts @@ -156,7 +156,15 @@ async function runValeEngine(options: DispatchOptions): Promise { }); if (outcome.status === "ok") { - return { engine: "vale", results: outcome.results }; + // A zero-exit run that still wrote to stderr carries a diagnostic — most + // often a rule assignment Vale ignored for sitting outside a section. It + // rides through as a notice and never as a failure: the run succeeded, and + // letting it touch the exit code would fail checks over a warning. + return { + engine: "vale", + results: outcome.results, + ...(outcome.notice === undefined ? {} : { notice: outcome.notice }), + }; } return outcome.blocking ? { engine: "vale", results: [], failure: outcome.message } diff --git a/packages/cli/src/rules/vale/run.ts b/packages/cli/src/rules/vale/run.ts index a252e5a..7c1fb09 100644 --- a/packages/cli/src/rules/vale/run.ts +++ b/packages/cli/src/rules/vale/run.ts @@ -55,9 +55,23 @@ export const VALE_TIMEOUT_MS = 60_000; * * `ok` is non-blocking even when it carries findings: severity decides the exit * code there, the same as for every other engine. + * + * `ok` also carries an optional `notice`: whatever Vale wrote to stderr while + * still exiting zero. That combination is not noise. Vale reports a rule + * assignment placed outside any section as `W101 … is ignoring it` — on stderr, + * with exit 0 and a well-formed empty result on stdout — so discarding it + * leaves an author with a rule that verifies, runs, and reports nothing. + * Surfacing it is what makes the section-less scaffold safe: the mistake it + * invites becomes legible instead of silent. */ export type ValeRunOutcome = - | { status: "ok"; blocking: false; results: CheckResult[] } + | { + status: "ok"; + blocking: false; + results: CheckResult[]; + /** Vale's stderr on a zero-exit run, when it wrote any. */ + notice?: string; + } | { status: "unavailable"; blocking: false; message: string } | { status: "timeout"; blocking: true; message: string } | { status: "failed"; blocking: true; message: string }; @@ -214,13 +228,23 @@ export async function runVale( return; } + // Exit was zero, so anything on stderr is a diagnostic about a run that + // otherwise succeeded — the `W101` ignored-assignment warning above all. + // Attached to every `ok` path so a diagnostic cannot be dropped by which + // branch happened to produce the (empty) results. + const diagnostic = stderrChunks.join("").trim(); + const notice = + diagnostic === "" + ? {} + : { notice: `Vale reported while running: ${diagnostic}` }; + const stdout = stdoutChunks.join("").trim(); if (stdout === "") { // Measured: Vale prints `{}` when it finds nothing, which parses and // maps to [] below. This branch is for a Vale that says nothing at all // — cheap insurance against JSON.parse("") reporting a clean run as a // failure. - settle({ status: "ok", blocking: false, results: [] }); + settle({ status: "ok", blocking: false, results: [], ...notice }); return; } @@ -250,6 +274,7 @@ export async function runVale( status: "ok", blocking: false, results: toValeCheckResults(parsed as ValeOutput), + ...notice, }); } catch (error) { settle({ diff --git a/packages/cli/test/anonymous-flag.test.ts b/packages/cli/test/anonymous-flag.test.ts index 52a20f6..4ee35d8 100644 --- a/packages/cli/test/anonymous-flag.test.ts +++ b/packages/cli/test/anonymous-flag.test.ts @@ -86,7 +86,7 @@ describe("--anonymous flag (per-command behavior matrix)", () => { // file validation. const result = await runCli(["rule", "create", "--anonymous", "-d", cwd]); expect(result.exitCode).not.toBe(0); - expect(result.stderr).toContain("taskless agent rule-create --anonymous"); + expect(result.stderr).toContain("taskless agent create-sg-rule"); }); it("with --json, emits the standardized envelope", async () => { @@ -115,7 +115,7 @@ describe("--anonymous flag (per-command behavior matrix)", () => { cwd, ]); expect(result.exitCode).not.toBe(0); - expect(result.stderr).toContain("taskless agent rule-improve --anonymous"); + expect(result.stderr).toContain("taskless agent improve-rule --anonymous"); }); }); diff --git a/packages/cli/test/help-extensions.test.ts b/packages/cli/test/help-extensions.test.ts index b715528..a8ca2fa 100644 --- a/packages/cli/test/help-extensions.test.ts +++ b/packages/cli/test/help-extensions.test.ts @@ -64,14 +64,25 @@ describe("taskless agent (no args)", () => { expect(result.stdout).toContain("Authoring recipes:"); for (const topic of [ "route", - "existing", - "static", - "remote", - "engine-selection", + "create-legacy-rule", + "create-sg-rule", + "create-vale-rule", + "create-runtime-rule", + "create-remote-rule", ]) { expect(result.stdout).toContain(topic); } }); + + it("no longer advertises the removed topic names", async () => { + const result = await runCli(["agent", "-d", cwd]); + // `static`/`existing`/`remote` were renamed and `engine-selection` was + // merged into `route`. Listing a name that resolves to nothing is worse + // than not listing it — the agent spends a fetch to learn it is gone. + for (const removed of ["engine-selection", " existing", " static"]) { + expect(result.stdout).not.toContain(removed); + } + }); }); describe("taskless agent ", () => { @@ -85,7 +96,14 @@ describe("taskless agent ", () => { await rm(cwd, { recursive: true, force: true }); }); - it.each(["route", "existing", "static", "remote", "engine-selection"])( + it.each([ + "route", + "create-legacy-rule", + "create-sg-rule", + "create-vale-rule", + "create-runtime-rule", + "create-remote-rule", + ])( "resolves the %s recipe without an unknown-topic error", async (topic) => { const result = await runCli(["agent", topic, "-d", cwd]); @@ -95,6 +113,31 @@ describe("taskless agent ", () => { expect(result.stderr).not.toContain("Unknown command"); } ); + + // D9: a reader who arrived at the wrong recipe should find that out in the + // first line, where recovery is a re-decision, rather than after authoring + // the wrong artifact. Fixed shape across all five so it is recognisable. + it.each([ + "create-legacy-rule", + "create-sg-rule", + "create-vale-rule", + "create-runtime-rule", + "create-remote-rule", + ])("opens %s with the orientation banner", async (topic) => { + const result = await runCli(["agent", topic, "-d", cwd]); + expect(result.stdout).toContain("## You are here"); + expect(result.stdout).toContain(`This is \`${topic}\`.`); + expect(result.stdout).toContain("`taskless agent route`"); + }); + + it.each(["existing", "static", "remote", "engine-selection", "rule-create"])( + "no longer resolves the removed topic %s", + async (topic) => { + const result = await runCli(["agent", topic, "-d", cwd]); + expect(result.exitCode).not.toBe(0); + expect(result.stderr).toContain("Unknown command"); + } + ); }); describe("taskless agent ", () => { @@ -109,15 +152,15 @@ describe("taskless agent ", () => { }); it("returns the canonical recipe for a known topic", async () => { - const result = await runCli(["agent", "rule-create", "-d", cwd]); + const result = await runCli(["agent", "create-remote-rule", "-d", cwd]); expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("# Topic: rule create"); + expect(result.stdout).toContain("# Topic: create-remote-rule"); expect(result.stdout).toContain("## Goal"); expect(result.stdout).toContain("## Steps"); }); it("interpolates %(CLI_VERSION)s in the recipe header", async () => { - const result = await runCli(["agent", "rule-create", "-d", cwd]); + const result = await runCli(["agent", "create-remote-rule", "-d", cwd]); // Should contain a version pattern, not the literal placeholder. // Guard against both the legacy mustache syntax and the current // sprintf-js named-arg syntax leaking through. @@ -127,7 +170,7 @@ describe("taskless agent ", () => { }); it("interpolates %(INPUT_SCHEMA)s for topics with a Zod input", async () => { - const result = await runCli(["agent", "rule-create", "-d", cwd]); + const result = await runCli(["agent", "create-remote-rule", "-d", cwd]); expect(result.stdout).not.toContain("{{INPUT_SCHEMA}}"); expect(result.stdout).not.toContain("%(INPUT_SCHEMA)s"); // Embedded schema includes the JSON Schema $schema URI @@ -182,42 +225,49 @@ describe("taskless agent --anonymous (variant lookup)", () => { await rm(cwd, { recursive: true, force: true }); }); - it("returns the .anonymous variant when one exists (rule create)", async () => { + it("returns the .anonymous variant when one exists (improve-rule)", async () => { const result = await runCli([ "agent", - "rule-create", + "improve-rule", "--anonymous", "-d", cwd, ]); expect(result.exitCode).toBe(0); // The anonymous recipe declares "(anonymous)" in its header - expect(result.stdout).toContain("# Topic: rule create (anonymous)"); + expect(result.stdout).toContain("(anonymous)"); }); - it("returns the .anonymous variant for rule improve", async () => { - const result = await runCli([ + it("falls back to the canonical recipe when no variant exists (check)", async () => { + const canonical = await runCli(["agent", "check", "-d", cwd]); + const anonymous = await runCli(["agent", "check", "--anonymous", "-d", cwd]); + expect(anonymous.exitCode).toBe(0); + // Same body — falls back to check.txt since no check.anonymous.txt + expect(anonymous.stdout).toBe(canonical.stdout); + }); + + // `rule-create.anonymous.txt` used to be the local-only variant of the + // service recipe, which duplicated `static.txt` outright. `create-sg-rule` + // now *is* that path, so an `--anonymous` variant of the remote recipe would + // be "the local version of the remote one" — the contradiction `route` + // resolves. It falls back to the canonical text instead. + it("has no anonymous variant of the service recipe", async () => { + const canonical = await runCli(["agent", "create-remote-rule", "-d", cwd]); + const anonymous = await runCli([ "agent", - "rule-improve", + "create-remote-rule", "--anonymous", "-d", cwd, ]); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("# Topic: rule improve (anonymous)"); - }); - - it("falls back to the canonical recipe when no variant exists (check)", async () => { - const canonical = await runCli(["agent", "check", "-d", cwd]); - const anonymous = await runCli(["agent", "check", "--anonymous", "-d", cwd]); expect(anonymous.exitCode).toBe(0); - // Same body — falls back to check.txt since no check.anonymous.txt expect(anonymous.stdout).toBe(canonical.stdout); + expect(anonymous.stdout).not.toContain("(anonymous)"); }); it("returns the canonical recipe when --anonymous is omitted", async () => { - const result = await runCli(["agent", "rule-create", "-d", cwd]); - expect(result.stdout).toContain("# Topic: rule create"); + const result = await runCli(["agent", "improve-rule", "-d", cwd]); + expect(result.stdout).toContain("# Topic: rule improve"); expect(result.stdout).not.toContain("(anonymous)"); }); }); @@ -233,52 +283,77 @@ describe("bare taskless (non-TTY) routes to the agent index", () => { }); }); -describe("taskless agent engine-selection", () => { +// D1: `engine-selection` was merged into `route`, so the engine reasoning it +// carried has to still be reachable — from one fetch instead of two. These are +// the assertions that used to guard the standalone topic, re-pointed at the +// recipe that now owns them. +describe("taskless agent route (absorbed engine reasoning)", () => { let cwd: string; beforeEach(async () => { - cwd = await mkdtemp(join(tmpdir(), "taskless-agent-engine-")); + cwd = await mkdtemp(join(tmpdir(), "taskless-agent-route-")); }); afterEach(async () => { await rm(cwd, { recursive: true, force: true }); }); - it("names all three engines and puts evidence before the answer", async () => { - const result = await runCli(["agent", "engine-selection", "-d", cwd]); + it("puts the evidence before the destination", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); expect(result.exitCode).toBe(0); - for (const engine of ["`sg`", "`vale`", "`runtime`"]) { - expect(result.stdout).toContain(engine); - } - expect(result.stdout).toContain("before naming an engine"); + expect(result.stdout).toContain("before you name a destination"); }); - it("carries the three boundary cases", async () => { - const result = await runCli(["agent", "engine-selection", "-d", cwd]); + it("carries the boundary cases", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); // Each is a wrong answer someone actually reaches for. expect(result.stdout).toContain("Prose about code is still prose"); expect(result.stdout).toContain("one document at a time"); - expect(result.stdout).toContain("Engine is not trust tier"); + expect(result.stdout).toContain("Trust tier is not a destination"); }); it("states the ambiguity default as a property, not as `sg`", async () => { - const result = await runCli(["agent", "engine-selection", "-d", cwd]); - // D7: "choose an engine you know is available" stays correct on an + const result = await runCli(["agent", "route", "-d", cwd]); + // "choose an engine whose availability you can assert" stays correct on an // unsupported arch and server-side alike; naming `sg` outright would be // false on the host where `sg` is the missing one. - expect(result.stdout).toContain("choose an engine you know is"); - expect(result.stdout).toContain("available"); + expect(result.stdout).toContain("whose availability you can"); + expect(result.stdout).toContain("There is no fixed fallback"); + }); + + it("weighs the concrete form above the wording of the request", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); + expect(result.stdout).toContain("outranks how the request was"); + }); + + it("names all five destinations", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); + for (const destination of [ + "create-legacy-rule", + "create-sg-rule", + "create-vale-rule", + "create-runtime-rule", + "create-remote-rule", + ]) { + expect(result.stdout).toContain(destination); + } + }); + + it("splits the runtime destination on login state", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); + // D7: reading login state early is what makes the destination set correct. + expect(result.stdout).toContain("info --json"); + expect(result.stdout).toContain("logged out"); }); - it("stays out of the authoring-destination decision", async () => { - const result = await runCli(["agent", "engine-selection", "-d", cwd]); - // Scope guard (3.4): it may point at `route`, never re-decide it. - expect(result.stdout).toContain("is `route`, and it is a separate"); + it("sends the reader to a command rather than a category", async () => { + const result = await runCli(["agent", "route", "-d", cwd]); + expect(result.stdout).toMatch(/agent create-vale-rule/); }); it("follows the recipe header and section convention", async () => { - const result = await runCli(["agent", "engine-selection", "-d", cwd]); - expect(result.stdout).toContain("# Topic: engine-selection"); + const result = await runCli(["agent", "route", "-d", cwd]); + expect(result.stdout).toContain("# Topic: route"); for (const section of [ "## Goal", "## Preconditions", diff --git a/packages/cli/test/prompts.test.ts b/packages/cli/test/prompts.test.ts index f7de904..9f365cd 100644 --- a/packages/cli/test/prompts.test.ts +++ b/packages/cli/test/prompts.test.ts @@ -30,12 +30,12 @@ const UNRESOLVED_PLACEHOLDER = /%\([A-Z_]+\)s/; * something the bundler or the type-checker resolves back to source. */ async function importBuiltPrompts(): Promise<{ - getPrompt: (topic: "static", options?: PromptOptions) => string; + getPrompt: (topic: "create-sg-rule", options?: PromptOptions) => string; TOPICS: readonly string[]; }> { const url = pathToFileURL(distributionPromptsPath).href; return (await import(/* @vite-ignore */ url)) as { - getPrompt: (topic: "static", options?: PromptOptions) => string; + getPrompt: (topic: "create-sg-rule", options?: PromptOptions) => string; TOPICS: readonly string[]; }; } @@ -62,12 +62,12 @@ describe("prompt rendering", () => { }); it("renders the CLI version into the header", () => { - expect(getPrompt("static")).toContain(`CLI v${__VERSION__}`); + expect(getPrompt("create-sg-rule")).toContain(`CLI v${__VERSION__}`); }); it.each([ - ["rule-create", "prompt"], - ["rule-improve", "ruleId"], + ["create-remote-rule", "prompt"], + ["improve-rule", "ruleId"], ])("renders the JSON Schema for %s", (topic, property) => { const rendered = getRecipe(topic) ?? ""; expect(rendered, `${topic} lost its schema`).not.toContain( @@ -95,10 +95,10 @@ describe("prompt rendering", () => { describe("header suppression", () => { it("drops the header line and the blank line after it, leaving the body intact", () => { - const withHeader = getPrompt("static"); - const withoutHeader = getPrompt("static", { header: false }); + const withHeader = getPrompt("create-sg-rule"); + const withoutHeader = getPrompt("create-sg-rule", { header: false }); - expect(withHeader.startsWith("# Topic: static")).toBe(true); + expect(withHeader.startsWith("# Topic: create-sg-rule")).toBe(true); expect(withoutHeader.startsWith("# Topic:")).toBe(false); // The body is the same string, minus the header line and its blank line. expect(withoutHeader).toBe(withHeader.split("\n").slice(2).join("\n")); @@ -117,22 +117,26 @@ describe("header suppression", () => { }); it("keeps the header by default", () => { - expect(getPrompt("static")).toBe(getPrompt("static", { header: true })); - expect(getPrompt("static")).toBe(getPrompt("static", {})); + expect(getPrompt("create-sg-rule")).toBe( + getPrompt("create-sg-rule", { header: true }) + ); + expect(getPrompt("create-sg-rule")).toBe(getPrompt("create-sg-rule", {})); }); }); describe("anonymous variants", () => { it("returns the variant text for a topic that has one", () => { - const canonical = getRecipe("rule-create"); - const anonymous = getRecipe("rule-create", { anonymous: true }); + const canonical = getRecipe("improve-rule"); + const anonymous = getRecipe("improve-rule", { anonymous: true }); expect(anonymous).toBeDefined(); expect(anonymous).not.toBe(canonical); expect(anonymous).toContain("(anonymous)"); }); it("falls back to the canonical recipe for a topic without one", () => { - expect(getRecipe("static", { anonymous: true })).toBe(getRecipe("static")); + expect(getRecipe("create-sg-rule", { anonymous: true })).toBe( + getRecipe("create-sg-rule") + ); }); }); @@ -145,8 +149,8 @@ describe("typed accessor", () => { }); it("passes options through the map", () => { - expect(PROMPTS.static({ header: false })).toBe( - getPrompt("static", { header: false }) + expect(PROMPTS["create-sg-rule"]({ header: false })).toBe( + getPrompt("create-sg-rule", { header: false }) ); }); @@ -220,7 +224,7 @@ describe("built prompts entry", () => { it("inlines every build define", async () => { const { getPrompt: getBuiltPrompt } = await importBuiltPrompts(); - const rendered = getBuiltPrompt("static"); + const rendered = getBuiltPrompt("create-sg-rule"); expect(rendered).toContain(`CLI v${__VERSION__}`); expect(rendered).not.toMatch(/__[A-Z_]+__/); expect(rendered).not.toMatch(UNRESOLVED_PLACEHOLDER); @@ -228,9 +232,9 @@ describe("built prompts entry", () => { it("renders identically from the built artifact and from source", async () => { const { getPrompt: getBuiltPrompt } = await importBuiltPrompts(); - expect(getBuiltPrompt("static")).toBe(getPrompt("static")); - expect(getBuiltPrompt("static", { header: false })).toBe( - getPrompt("static", { header: false }) + expect(getBuiltPrompt("create-sg-rule")).toBe(getPrompt("create-sg-rule")); + expect(getBuiltPrompt("create-sg-rule", { header: false })).toBe( + getPrompt("create-sg-rule", { header: false }) ); }); }); From d24864d7178f81af6ecebf3c5990d69a5c3a1c8a Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 13:38:09 -0700 Subject: [PATCH 09/19] docs(openspec): rewrite the resume notes for group 2 and the 2b handoff --- .../resume.md | 213 ++++++++++++------ 1 file changed, 141 insertions(+), 72 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/resume.md b/openspec/changes/agent-command-and-vale-authoring/resume.md index c07e65f..64c5a47 100644 --- a/openspec/changes/agent-command-and-vale-authoring/resume.md +++ b/openspec/changes/agent-command-and-vale-authoring/resume.md @@ -1,111 +1,180 @@ # Resume notes — `agent-command-and-vale-authoring` -Handoff for picking this up after a context reset. Not part of the OpenSpec artifact set; -delete it when the change is archived. +Handoff for picking this up after a context reset, or on another machine. Not part of +the OpenSpec artifact set; delete it when the change is archived. ## Where you are - **PR #102** (draft), branch `openspec/agent-command-and-vale-authoring`, stacked on **#100**. -- **Work in the worktree** at `worktrees/impl-102`, not the main checkout. The main checkout - should be on `main`; a branch can only be checked out in one worktree at a time. - Stack below you, all green and already reviewed: **#71 → #93 → #94 → #95 → #100**. The user drives the merge-down; do not merge anything without being asked. -- **Group 1 of 6 is complete** (`093aca1`). Everything else is unstarted. - `tasks.md` is the authority — read it first, and trust its checkboxes over this file. +- **Groups 1, 2, 5.1 and 5.2 are done** (`093aca1`, `b0a0ee0`). 572 tests pass, typecheck + and lint clean. `tasks.md` is the authority — read it first and trust its checkboxes + over this file. +- Work happens in the worktree at `worktrees/impl-102`, not the main checkout. On a fresh + machine: `git worktree add worktrees/impl-102 openspec/agent-command-and-vale-authoring` + then **`pnpm install` inside it** — a worktree gets its own empty `node_modules`, and + skipping the install breaks `git commit` (lint-staged) and every `pnpm` script. ## Read these before doing anything 1. `tasks.md` — the plan, including group **2b** (the recipe test harness) -2. `design.md` — decisions D1–D10, each with its rejected alternatives. All open questions closed. +2. `design.md` — decisions D1–D10, each with its rejected alternatives. No open questions. 3. `specs/*/spec.md` — what the recipes and the scaffold must do 4. `proposal.md` — the why, and the delivery shape (single PR, stacked on #100) ## Environment traps that will cost you an hour each -- **`NODE_OPTIONS` is broken in this session's shell.** It carries a `--require` preload - pointing at a deleted temp file, so every `node`, `pnpm`, and `git commit` (husky) call - dies with `MODULE_NOT_FOUND`. Prefix everything: - `NODE_OPTIONS="--max-old-space-size=4096"`. The symptom is a build that looks like it - exits 0 through a pipe while `dist/` is never written, which then fails ~125 unrelated tests. +- **`NODE_OPTIONS` was broken in the previous session's shell** — a `--require` preload + pointing at a deleted temp file, so every `node`, `pnpm`, and `git commit` died with + `MODULE_NOT_FOUND`. Every command in that session was prefixed with + `NODE_OPTIONS="--max-old-space-size=4096"`. **On a fresh machine, check whether you + still need this** (`echo $NODE_OPTIONS`) rather than cargo-culting it. - **Run `pnpm --filter @taskless/cli build` before `pnpm --filter @taskless/cli test`.** - Many suites spawn the built CLI. A stale or missing `dist/` produces failures that read - exactly like real regressions — this already caused one false alarm on `main`. -- **`commit.gpgsign` is true locally.** Keep it. An earlier restack silently stripped + Many suites spawn the built CLI. A stale `dist/` produces failures that read exactly + like real regressions. +- **`commit.gpgsign` must be true locally.** An earlier restack silently stripped signatures from 34 commits because `git rebase` does not re-sign without it, and CI does - not catch unsigned commits. If you rebase, audit with `git log --format='%G?'` afterwards. -- **Do not use `git checkout ` to undo a probe** when the file also holds uncommitted - work — it discards both. This bit twice in the previous session. - -## Next up: group 2, and how to do it - -Group 2 is the authorship — the part that needs care. The user wants to see the prose before -it is finalised, but **only after 2b has been run against it**, so they review a tested -artifact rather than a draft. - -Order that works: - -1. Draft `create-vale-rule.txt`. Source of truth for Vale specifics is - `https://docs.vale.sh/llms-full.txt` (a full corpus for LLM consumption), or - `https://docs.vale.sh/topics/.md?ask=`. **`https://docs.vale.sh/styles` 404s.** - Known-good facts already measured this session: - - Common rule fields: `extends` (required), `message` (required), - `level` (`suggestion`|`warning`|`error`, default `suggestion`), `scope`, `link`. - - Eleven extension points: `existence`, `substitution`, `occurrence`, `repetition`, - `consistency`, `conditional`, `capitalization`, `metric`, `spelling`, `sequence`, `script`. - - `StylesPath = .` makes `rules/` the StyleName, so a rule at `vale/rules/no-simply.yml` - has the check id `rules.no-simply`. `StylesPath = rules` resolves nothing — measured. - - A rule assignment outside any `[section]` is **silently ignored**; Vale reports - `W101 ... is ignoring it` on **stderr** with exit 0 and a valid `{}` on stdout. -2. Model its shape on `create-sg-rule.txt` (the renamed `static.txt`). If the Vale recipe - diverges structurally, that is a signal you invented something rather than followed the - established pattern. -3. **Run group 2b against it** before showing the user anything. See `tasks.md` 2b.1–2b.8. - The load-bearing constraint: the test subagent gets the recipe text, a sandbox path, and a - rule intent — and **no repository access**. With repo access it finds the existing - `no-simply.yml` and the mixed-engine fixture and copies them, and you are testing fixtures - rather than prose. -4. Bring the user the recipe **plus the iteration log**. - -## Group 2 landmines already identified - -- **`help-extensions.test.ts` has a five-test `taskless agent engine-selection` block.** - Task 2.7 deletes that topic, so those tests must be retired as part of the merge, not after. -- **`RECIPE_TOPICS` in `agent.ts`** still lists `route/existing/static/remote/engine-selection`. - Both `help-extensions.test.ts` and `help-routing-telemetry.test.ts` assert against that exact - list. Groups 2 and 4 both touch it. -- **The `remote` + `rule-create` merge (2.5a) is a content merge, not a rename.** Both texts - have material that survives; the result must read as one procedure rather than two - concatenated. This is the piece most worth the user's eye. -- Group 1 deliberately left recipe `.txt` files untouched. **Do not start sweeping - cross-references during group 2** — that is 3.1, and mixing them makes both unreviewable. + not catch unsigned commits. Audit with `git log --format='%G?'` after any rebase. +- **A recipe containing a literal `%` must escape it as `%%`.** Recipes render through + sprintf-js named args, so a bare `%s` in prose (Vale's `message:` examples are full of + them) fails at render with "mixing positional and named placeholders is not supported". + It is caught by rendering the topic, not by the build or by typecheck. +- **`zsh` mangles `perl -0pi -e` one-liners containing `@`.** Use `python3 - <<'PY'` for + multi-file text surgery; two attempts were lost to quoting before switching. + +## What group 2 actually produced + +Topic map after the rename, so you do not have to reconstruct it from the diff: + +| Before | After | +| --------------------------- | ------------------------- | +| `static.txt` | `create-sg-rule.txt` | +| `existing.txt` | `create-legacy-rule.txt` | +| `remote.txt` + `rule-create.txt` | `create-remote-rule.txt` (content merge) | +| `engine-selection.txt` | merged into `route.txt`, deleted | +| `rule-create.anonymous.txt` | **deleted** (see below) | +| `rule-improve*.txt` | `improve-rule*.txt` | +| `rule-delete.txt` | `delete-rule.txt` | +| `rule-verify.txt` | `verify-rule.txt` | +| — | `create-vale-rule.txt` (new) | +| — | `create-runtime-rule.txt` (new) | +| `rule.txt`, `rule-meta.txt` | unchanged names | + +**The one deviation from the task text**: `rule-create.anonymous.txt` was deleted rather +than renamed to `create-remote-rule.anonymous.txt`. It duplicated `static.txt` outright, +and "the local-only variant of the remote recipe" is the contradiction `route` exists to +resolve. Its unique material (upstream-schema pointer, optional fields, the per-layer +verify error table) moved into `create-sg-rule.txt`, and `rule create --anonymous` now +points at `taskless agent create-sg-rule`. Recorded in `tasks.md` 2.3. + +**5.1 and 5.2 were pulled forward**, out of group order and deliberately: 2b tests +`create-vale-rule` against a scaffolded project, and the recipe's central claim is that +the scaffold ships section-less. Testing against a scaffold that still wrote `[*]` would +have exercised a recipe nobody will receive. + +## Next up: finish 2b (the harness), then 5.3/5.4, then group 3 + +### 2b is mid-flight + +Three sandboxed subagents were launched to author a real Vale rule from the rendered +recipe and had **not reported back** when the session ended. Their sandboxes are under +the scratchpad at `vale-harness/sandbox-{a,b,c}`, each a real `init` scaffold, alongside +`create-vale-rule.rendered.txt` (the dev-build render they were given). **That scratchpad +is machine-local — on another machine, re-run the harness from scratch rather than looking +for it.** + +To re-run it: + +1. `pnpm --filter @taskless/cli build:dev` — `TASKLESS_BUILD_TARGET=dev` bakes + `__TASKLESS_CLI__` as an **absolute** path to `dist-dev/index.js`, so the rendered + recipe carries a command that runs from any directory. Confirmed working: the render's + step-5 commands come out as `node /abs/path/dist-dev/index.js check …`. +2. Scaffold a throwaway project with that binary (`init --no-interactive -d `), + so the sandbox is a real scaffold and not a hand-made approximation. +3. Render the recipe to a file (`agent create-vale-rule > …rendered.txt`). +4. Hand a **fresh, non-forked** subagent only: the rendered recipe path, the sandbox path, + and a rule intent in plain words. It must **not** have repository access — with it, it + finds `no-simply.yml` and the mixed-engine fixture and copies them, and the loop tests + our fixtures rather than our writing. Ask explicitly for a blunt critique of the prose; + that is the deliverable, not the rule. +5. The three intents used, chosen to exercise different extension points (everything in + this repo today is `existence`, so a recipe drafted from our own examples teaches token + blocklists and nothing else): hedging phrases (`existence`), "sign in" vs "login" + (`substitution`), and GitHub's capitalization (`capitalization`, literal-match form — + the variant the recipe covers in one line). +6. Every failure is a defect in the prose. Fix the recipe, re-run with a fresh agent. + Converged when an agent produces a rule that fires on `fail/` and stays quiet on + `pass/`, first try, uncorrected. Keep the iteration log (2b.7) — it is the only part a + reviewer can check without rerunning the loop. +7. 2b.8: run the same harness over `create-sg-rule` as a control. A failure there means + the harness is wrong rather than the recipe. + +### Facts measured this session — do not re-derive + +- Vale field reference for the three taught extension points is in the recipe and came + from `https://docs.vale.sh/llms-full.txt`. **`https://docs.vale.sh/styles` is fine (200)** + — the earlier note that it 404s was wrong; it needs `curl -L`. +- `StylesPath = .` makes `rules/` the StyleName, so `vale/rules/no-simply.yml` is the + check `rules.no-simply`. `StylesPath = rules` resolves nothing. +- **`BasedOnStyles =` is not required for a rule to fire, and omitting it added no noise** + in the bundled Vale version. The recipe still tells authors to write it, on the honest + grounds that it is explicit and matches what `verify` generates — not on the claim that + omitting it produces spurious findings, which measurement did not support. +- A rule assignment outside any section: `W101 '' isn't a core option; Vale is + ignoring it` on **stderr**, exit 0, valid `{}` on stdout. With 5.2 this now surfaces as + `Notice: Vale reported while running: …` and the check still exits 0. Verified end to + end against the built CLI. + +### A real gap found, not yet decided + +**`verifyValeRule` / `verifyValeRules` have no CLI caller.** They are exported from +`src/rules/vale/verify.ts` and exercised only by `test/vale-verify.test.ts`; +`taskless rule verify ` routes to `src/rules/verify.ts`, which is ast-grep only. So +there is currently **no way for an agent to verify a Vale rule from the CLI**. + +`create-vale-rule` works around this by validating with `check` over each fixture bucket, +which does work today and is what the harness exercises. But it means task **2b.4's** +"`verify` passes" cannot be satisfied as written, and it is the same class of dead end +this whole change exists to remove — a capability that exists but is unreachable. + +Wiring it looks small: dispatch `rule verify ` by which engine owns the id +(`.taskless/vale/rules/.yml` vs `.taskless/sg/rules/.yml`), then call +`verifyValeRule`. **Raise this with the user before doing it** — it is scope not in +`tasks.md`, and the alternative (a follow-up issue, like #99 and #101) is defensible. ## Group 3 note, when you get there `rule.txt` documents a table of multi-token forms (`taskless help rule create`, -`... rule meta`). After group 1 these are **actively broken**, not merely stale — they hit the -new "Too many arguments" path. Lead group 3 with it. +`… rule meta`). After group 1 these are **actively broken**, not merely stale — they hit +the "Too many arguments" path. Lead group 3 with it. + +Group 2 deliberately left the wider cross-reference sweep alone (~306 `taskless help` +occurrences across 77 files). It did update See Also blocks in the six recipes it +rewrote, so 3.1 is the remaining files plus `skills/taskless/SKILL.md`, both READMEs, and +the TS sources. Leave `CHANGELOG.md` alone. ## Decisions you should not silently revisit All are argued in `design.md` with rejected alternatives. The two most likely to be re-litigated by accident: -- **D1** — `route` and `engine-selection` merge; the engine criterion is stated **once**, in - `route`'s destination table. Destinations carry a short orientation line (D9), never a second - copy of the criterion. +- **D1** — `route` and `engine-selection` merge; the engine criterion is stated **once**, + in `route`'s destination table. Destinations carry a short orientation line (D9), never + a second copy of the criterion. `test/help-extensions.test.ts` now guards both halves. - **D4** — the section-less scaffold ships **paired** with surfacing Vale's stderr on a - zero-exit run. Shipping the scaffold alone reintroduces the silent-disable failure the whole - Vale stack exists to eliminate. They are one requirement, not two. + zero-exit run. Shipping the scaffold alone reintroduces the silent-disable failure the + whole Vale stack exists to eliminate. They are one requirement, not two. Both are now in. -Also settled: pre-1.0, every backwards-incompatible change here is a **MINOR** bump — never -MAJOR. The telemetry event stays `cli_help` (the user wants agent-call volume visible under the -existing event). +Also settled: pre-1.0, every backwards-incompatible change here is a **MINOR** bump — +never MAJOR. The telemetry event stays `cli_help` (agent-call volume stays visible under +the existing event). ## Outside this PR - **#99** — migrate subprocess handling to execa (inventory and sequencing already written up) - **#101** — whether a whole-project Vale check should skip build output; `.taskless/` is already excluded as of #100 -- Stale worktrees under `worktrees/` from earlier agents; `git worktree list` to review +- Stale worktrees under `worktrees/`; `git worktree list` to review - `openspec validate --all --strict` fails on `spec/cli-rules` and `spec/cli-update-engine` on `main` already — pre-existing, unrelated, do not chase it From 7fce2ec353b2c9de09ac9db11f20e9ac31cbae87 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 13:42:56 -0700 Subject: [PATCH 10/19] docs(openspec): record the first 2b harness run and its triage --- .../iteration-log.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 openspec/changes/agent-command-and-vale-authoring/iteration-log.md diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md new file mode 100644 index 0000000..a82b867 --- /dev/null +++ b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md @@ -0,0 +1,82 @@ +# 2b iteration log — executing the authoring recipes + +Task 2b.7: the record of what failed, what changed, and what finally held. The recipes are +the deliverable, and a recipe that reads well to its author while producing the wrong +artifact is exactly what reviewing the prose cannot catch — so they get executed. + +Delete this file when the change is archived. + +## Harness + +- `pnpm --filter @taskless/cli build:dev`, so `__TASKLESS_CLI__` is an absolute path to + `dist-dev/index.js` and the rendered recipe carries a command that runs from anywhere. +- Sandbox is a real `init --no-interactive` scaffold, not a hand-made approximation. +- Each run gets a **fresh, non-forked** subagent handed exactly three things: the rendered + recipe, the sandbox path, and a rule intent in plain words. **No repository access** — + with it, the agent finds `no-simply.yml` and the mixed-engine fixture and copies them, + and the loop tests our fixtures rather than our writing. +- Intents chosen to exercise different extension points. Everything in this repo today is + `existence`, so a recipe drafted from our own examples would teach token blocklists and + nothing else. + +## Round 1 + +| Run | Intent | Extension point | Converged | +| --- | ------ | --------------- | --------- | +| A | flag hedging phrases in docs | `existence` | **yes, first try, zero retries** | +| B | "sign in" is the verb, "login" the noun | `substitution` | pending | +| C | it is `GitHub`, not `Github`/`github` | `capitalization` (literal match) | pending | + +### Run A — converged, and that is the problem + +The agent produced `no-hedging.yml`, added a scoped `[*.md]` section with +`BasedOnStyles =` and `rules.no-hedging = YES`, wrote both fixture buckets, and got six +findings on `fail/` and zero on `pass/` on the first execution. No `W101` notice, so the +assignment landed inside the section. + +That is the 2b.6 bar met for one extension point. It also means **the step-5 debug ladder +was never exercised**, and the agent's critique is mostly about the rungs that ladder is +missing. Taking a first-try pass as evidence the prose is finished would be reading the +result backwards. + +Findings, triaged. Verified against the source rather than taken on the agent's word: + +| # | Finding | Verdict | +| - | ------- | ------- | +| a | `--json` reports `success: true` and exit 0 on **both** buckets; only `results.length` distinguishes them. The recipe hands you `--json` and describes no field of it | **Real.** An agent checking `$?` or grepping `success` concludes a working rule failed, and starts debugging it | +| b | A whole-project `check` reports nothing from the fixtures, contradicting step 3's walk model and step 6's suggestion | **Real.** `run.ts:137` excludes `.taskless/**` from the whole-project walk (the #100 fix). Fixtures live under `.taskless/`, so they are invisible to a bare `check` — correct behavior, undocumented, and the recipe's own model mispredicts its own suggested command | +| g | Step 4 asserts the CLI rejects a nested fixture directory and requires both buckets — neither is reachable | **Real, and the worst of these.** That validation lives in `verifyValeRule`, which has **zero CLI callers**. The recipe states an invariant nothing enforces, so an author who omits `pass/` sails through step 5 and never learns | +| d | `MinAlertLevel` is absent from the debug ladder, though `MinAlertLevel = error` against a `level: warning` rule produces exactly the silent failure the ladder exists for, with every listed item checking out | **Real** | +| c | `tokens` are regexes; the recipe presents them as a word list and never mentions `raw` or `nonword`, or what happens to a phrase containing regex metacharacters | **Real.** Live for hedging phrases specifically ("maybe?") | +| i | `%s` count is a property of the extension point — one for `existence`, two for `substitution` — stated nowhere, so copying the field table into a substitution rule yields `%!s(MISSING)` | **Real, cheap to fix** | +| f | `StylesPath = .` is never said to resolve relative to the directory holding `.vale.ini`, and the recipe uses two path roots (`.taskless/vale/rules/…` vs `vale/rules/…`) without reconciling them | **Real** | +| h | Three names must agree — filename, `id`, fixture directory — and are explained as if they were one | **Real** | +| e | Step 5 gives a runnable `node … check` while step 6 and See Also give `taskless agent check`; the agent could not tell whether `agent check` is a different subcommand | **Real, and corpus-wide.** Every recipe uses the bare `taskless agent ` form for a fetch and `npx @taskless/cli ` for an invocation. Obvious to us, not to a first-time reader | +| j | Fenced code blocks inside markdown: does a rule fire inside them? Undefined here, and live for a docs rule | **Real, needs measuring** before writing an answer | +| k | Step 1 sends the reader to `docs.vale.sh` with no offline fallback for the eight extension points it declines to describe | Accepted. Embedding the full Vale reference is out of scope | + +Nothing in this list is a defect in the agent. Every one is a defect in the prose, which +is what 2b.6 says to treat them as. + +## Planned revision (not yet applied) + +Batched until B and C report, so the recipe is revised once against all three rather than +three times against one. + +1. **Step 5 — say how to read the output.** `results` non-empty on `fail/`, empty on + `pass/`; `success` and the exit code are the same in both and are not the signal. +2. **Step 5/6 — fixtures are excluded from a whole-project `check`.** Say so, and stop + implying a bare `check` is a way to see the rule fire on its own fixtures. +3. **Step 4 — stop asserting validation nothing performs.** Either the invariants become + reachable (wire `rule verify ` to dispatch by engine — see `resume.md`, needs the + user's call) or the recipe states them as author discipline rather than as something + the CLI enforces. Do not leave the current wording; it is false. +4. **Step 5 — add `MinAlertLevel` to the debug ladder**, above the pattern. +5. **Step 2 — `tokens` are regexes.** Name `raw` and `nonword`, and say metacharacters are + live. +6. **Step 2 — `%s` count follows the extension point.** +7. **Step 3 — `StylesPath` resolves relative to the config file**, and use one path root. +8. **Step 2/3 — the three names that must agree**, said once as one fact. +9. **Measure the fenced-code-block question**, then answer it in a line. + +Item 3 is the one that needs a decision rather than a wording pass. From 46caa85ed64df753c3189fe9cf8a72ba9790351f Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 13:43:09 -0700 Subject: [PATCH 11/19] docs(openspec): point the resume notes at the 2b iteration log --- .../agent-command-and-vale-authoring/resume.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/resume.md b/openspec/changes/agent-command-and-vale-authoring/resume.md index 64c5a47..db8ac97 100644 --- a/openspec/changes/agent-command-and-vale-authoring/resume.md +++ b/openspec/changes/agent-command-and-vale-authoring/resume.md @@ -75,10 +75,18 @@ have exercised a recipe nobody will receive. ## Next up: finish 2b (the harness), then 5.3/5.4, then group 3 -### 2b is mid-flight - -Three sandboxed subagents were launched to author a real Vale rule from the rendered -recipe and had **not reported back** when the session ended. Their sandboxes are under +### 2b is mid-flight — read `iteration-log.md` first + +**Run A has reported and is written up in `iteration-log.md`**, with its findings triaged +against the source and a nine-item planned revision. It converged first try, which is +worth reading skeptically: the step-5 debug ladder went untested, and the findings are +mostly about rungs that ladder is missing. **Item 3 of the planned revision needs a +decision, not a wording pass** — the recipe currently asserts fixture validation that no +CLI command performs (see "A real gap" below). + +Runs B (`substitution`) and C (`capitalization`) had **not reported** when the session +ended. The revision is deliberately batched until all three are in, so the recipe is +revised once against three reports rather than three times against one. Their sandboxes are under the scratchpad at `vale-harness/sandbox-{a,b,c}`, each a real `init` scaffold, alongside `create-vale-rule.rendered.txt` (the dev-build render they were given). **That scratchpad is machine-local — on another machine, re-run the harness from scratch rather than looking From 481a0258da1a070789becb54d6bec014982ebb85 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 13:58:02 -0700 Subject: [PATCH 12/19] docs(openspec): record the second 2b harness run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run B (substitution) converged first try and independently confirmed run A's three worst findings. Its most valuable addition: both `tokens` and `swap` keys are Go RE2 patterns presented in the recipe as literal strings, and a wrong guess there fails as a silent non-match — the exact failure the recipe spends a section warning about. Also records one finding rejected as a harness artifact, so a later reader does not "fix" build:dev's absolute-path rewriting. --- .../iteration-log.md | 50 +++++++++++++++++-- 1 file changed, 46 insertions(+), 4 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md index a82b867..924059c 100644 --- a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md +++ b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md @@ -24,7 +24,7 @@ Delete this file when the change is archived. | Run | Intent | Extension point | Converged | | --- | ------ | --------------- | --------- | | A | flag hedging phrases in docs | `existence` | **yes, first try, zero retries** | -| B | "sign in" is the verb, "login" the noun | `substitution` | pending | +| B | "sign in" is the verb, "login" the noun | `substitution` | **yes, first try, zero retries** | | C | it is `GitHub`, not `Github`/`github` | `capitalization` (literal match) | pending | ### Run A — converged, and that is the problem @@ -58,6 +58,38 @@ Findings, triaged. Verified against the source rather than taken on the agent's Nothing in this list is a defect in the agent. Every one is a defect in the prose, which is what 2b.6 says to treat them as. +### Run B — converged, and independently confirms A's three worst findings + +The agent produced `login-as-verb.yml` using regex `swap` keys, scoped it to `[*.md]`, +wrote both buckets, and got four findings on `fail/` and zero on `pass/` first try. It +also volunteered an extra probe — appending a sentence to `pass/` to test a false positive +it suspected in the `to login` key — and re-ran the bucket. That is the behavior the +recipe wants and does not currently ask for. + +Two runs, no shared context, and both independently reported **a**, **g**, and the +fetch-versus-invoke ambiguity. Those are not taste. + +New findings on top of run A: + +| # | Finding | Verdict | +| - | ------- | ------- | +| l | `swap` keys are **Go RE2** regexes — `(?:…)` works, lookahead and lookbehind do not. The only example is two literal strings, so the agent guessed. A wrong guess fails as a **silent non-match**, the exact failure the recipe spends a section warning about | **Real, and the most valuable finding of the round.** Generalizes A's finding (c): both `tokens` and `swap` keys are patterns presented as literals | +| m | `%s` **ordering** in a substitution message is asserted only by an example that reads correctly under either interpretation. First `%s` is the swap value, second is the matched text — confirmed only by running the tool | **Real.** Pairs with A's finding (i) on `%s` count | +| n | Overlapping `swap` keys have undefined precedence. `can login` won over `login with`; `to logout` over `logout of`. First-alternative-wins is fine, but an author enumerating alternatives cannot tell how many findings a sentence yields, or which message | **Real** | +| o | Step 1 says "eleven in total" and the table lists eight; the other three are named in the following paragraph, behind a URL. Eleven only via arithmetic across two paragraphs | **Real, trivial.** List all eleven | +| p | `check ` lints everything under the path against the **whole config** — it is not scoped to the rule under test. With a second rule whose glob matches, the fail bucket reports both, and the recipe gives no vocabulary for that. Step 3 hints at the real isolation mechanism and never connects it to step 5 | **Real.** "Run the rule against each bucket" overstates what the command does | +| q | `BasedOnStyles =` with an empty right-hand side: the recipe insists on it without saying whether an empty value is valid INI to Vale | Minor. Measured: valid, no warning. Worth one clause | + +**One finding rejected as a harness artifact, recorded so nobody "fixes" it:** run B +objected that step 5 hardcodes an absolute path into someone's checkout +(`node /Users/…/dist-dev/index.js check …`). That is `build:dev` doing its job — it +rewrites `npx @taskless/cli` to an absolute path precisely so the harness command runs +from any directory. The shipped recipe says `npx @taskless/cli`. No change. + +Run B's second half of that objection is **not** an artifact and stands: the document +uses `npx @taskless/cli ` for invocations and `taskless agent ` for fetches +without ever explaining the relationship. That is finding **e**. + ## Planned revision (not yet applied) Batched until B and C report, so the recipe is revised once against all three rather than @@ -72,9 +104,19 @@ three times against one. user's call) or the recipe states them as author discipline rather than as something the CLI enforces. Do not leave the current wording; it is false. 4. **Step 5 — add `MinAlertLevel` to the debug ladder**, above the pattern. -5. **Step 2 — `tokens` are regexes.** Name `raw` and `nonword`, and say metacharacters are - live. -6. **Step 2 — `%s` count follows the extension point.** +5. **Step 2 — patterns, not literals.** Both `tokens` and `swap` keys are **Go RE2** + regexes: `(?:…)` works, lookahead and lookbehind do not, and metacharacters in a real + phrase are live. Name `raw` and `nonword`. State whether word boundaries are applied, + per extension point. Say that overlapping alternatives resolve first-wins. This one + item now carries findings c, l and n, and is the round's biggest single change. +6. **Step 2 — `%s` count *and* order follow the extension point.** One for `existence`, + two for `substitution`, and in a substitution the first is the swap value, the second + the matched text. +6a. **Step 1 — list all eleven extension points in the table**, rather than eight plus + three in prose behind a URL. +6b. **Step 5 — `check ` is not scoped to the rule under test.** It lints everything + under the path against the whole config. Say so, rather than "run the rule against + each bucket". 7. **Step 3 — `StylesPath` resolves relative to the config file**, and use one path root. 8. **Step 2/3 — the three names that must agree**, said once as one fact. 9. **Measure the fenced-code-block question**, then answer it in a line. From 1aeb157f88d81f2f130206ac227d251d7603d9f1 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 14:12:12 -0700 Subject: [PATCH 13/19] fix(cli): correct create-vale-rule against the harness findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 1 of 2b: three sandboxed agents authored a real Vale rule from the rendered recipe with no repository access. Two converged first try; the third took a retry and found the two defects that mattered. The field table was factually wrong. It said `%s` interpolates the match; for `substitution` a single `%s` interpolates the *replacement*, so "Use GitHub not %s" against "Github" rendered "Use GitHub not GitHub". This is the worst kind of defect — it passes every check the recipe tells you to run, and only a human reading the message sees it. The extension-point table also pointed product names at `capitalization`, which applies `match` to a whole scope and cannot express "this word, wherever it appears". Measured, it flags entire sentences. Product-name spelling is a `substitution`. Beyond those: tokens and swap keys are Go RE2 patterns presented as literals (no lookaround, implicit word boundaries, live metacharacters, first-wins on overlap); `success` and the exit code do not answer the question the fixtures ask, only `results[].ruleId` does; `check ` is not scoped to the rule under test; MinAlertLevel was missing from the debug ladder, which had no branch for the over-broad case at all; and the pass bucket is near-misses, not correct prose. Claims of CLI enforcement are removed rather than softened. Nothing validates the fixture layout today, and saying otherwise taught a reader to rely on a check that does not run. One agent recommendation was rejected on measurement: `ignorecase: true` does not flag text already equal to its replacement, so the proposed warning would have been false. 572 tests pass. Round 2 is running against the revised text. --- .../iteration-log.md | 109 ++++++++++- packages/cli/src/help/create-vale-rule.txt | 180 +++++++++++++----- 2 files changed, 240 insertions(+), 49 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md index 924059c..88e277c 100644 --- a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md +++ b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md @@ -25,7 +25,7 @@ Delete this file when the change is archived. | --- | ------ | --------------- | --------- | | A | flag hedging phrases in docs | `existence` | **yes, first try, zero retries** | | B | "sign in" is the verb, "login" the noun | `substitution` | **yes, first try, zero retries** | -| C | it is `GitHub`, not `Github`/`github` | `capitalization` (literal match) | pending | +| C | it is `GitHub`, not `Github`/`github` | ended up `substitution` | **one retry** | ### Run A — converged, and that is the problem @@ -90,7 +90,112 @@ Run B's second half of that objection is **not** an artifact and stands: the doc uses `npx @taskless/cli ` for invocations and `taskless agent ` for fetches without ever explaining the relationship. That is finding **e**. -## Planned revision (not yet applied) +### Run C — the only run that failed, and it found the worst defect + +The one intent that did not converge first try, and the one that earned the round. It +also declined the extension point the recipe recommended, which is why it succeeded. + +**C1 — the field table was factually wrong, and it fails silently.** The table said +"`%s` interpolates the match". For `substitution` that is false. Measured directly: + +``` +swap: {Github: GitHub}, message: "Use GitHub not %s", document text: "Github" +→ "Use GitHub not GitHub" (matchedText: "Github") +``` + +A single `%s` interpolates the **replacement**. The correct form takes two, filling +`(replacement, match)`. The recipe's own example block had it right while the normative +table one paragraph above said the opposite — so an author who reads the table rather than +copy-pasting the example ships a nonsense message. This is the worst defect found in the +round because it **passes every check the recipe tells you to run**: the rule fires, both +fixtures behave, the exit code is right, and only a human reading the message notices. + +**C2 — the extension-point table pointed at a check that cannot do the job.** The row read +"how something is capitalized (headings, **product names**)" → `capitalization`, and a +later line endorsed a literal `match` "for a product name". Measured, `match: GitHub`: + +``` +findings: 2 + 'Working with Github should be GitHub' matched: 'Working with Github' + 'We host on Github and it is fine. should be GitHub' matched: 'We host on Github and it is fine.' +``` + +`capitalization` applies `match` to a whole **scope** — a heading, a sentence — so it +flags entire sentences and cannot express "this word, wherever it appears". Product-name +spelling is a `substitution`. The recipe named the one use case in that row the check +cannot serve, then reinforced it two lines later. The agent only avoided the trap because +it already knew the check was scope-shaped. + +**C3 — the `pass/` bucket framing under-tests.** Step 4 justified the pass bucket as "the +half that catches an over-broad pattern" but described it as prose that is *correct*. +Correct prose proves nothing; the rule was never going to fire on it. The agent had to +build a throwaway probe file — URLs, code spans, `GITHUB_TOKEN` — because the fixture +model had no place for near-misses. + +**C4 — the debug ladder is one-sided.** Every rung addresses `fail/` reporting nothing. +Nothing addresses `pass/` firing, which is the over-broad case the pass bucket exists for. + +**One of run C's recommendations was rejected on measurement.** It proposed warning that +`ignorecase: true` would make the key `Github` also flag the correct `GitHub`. Measured: + +``` +ignorecase: true, swap {Github: GitHub}, text "Wrong github and Github here. Correct GitHub here." +→ 2 findings: 'github', 'Github' ('GitHub' NOT flagged) +``` + +Vale skips a match that already equals its replacement, so `ignorecase: false` is not +needed to protect the correct spelling. Writing that warning in would have taught +something false. The recipe states the measured behavior instead. Worth noting as the +round's reminder that an agent's diagnosis is a lead, not a finding. + +## Round 1 outcome + +Two of three converged first try; the third took one retry and produced the two findings +that mattered most. Against 2b.6 — "an intent the recipe never names, first try, +uncorrected" — the recipe **passed for `existence` and `substitution` and failed for the +product-name case**, which is the honest reading. + +Sixteen findings, fifteen accepted, one rejected on measurement. Every accepted finding is +a defect in the prose, which is what 2b.6 says to treat them as. + +## Revision applied + +`create-vale-rule.txt` was rewritten against all three reports (200 → 286 lines). What +changed, beyond the wording items: + +- **All eleven extension points** are in the table, with `capitalization` explicitly + scoped to "a whole heading or sentence" and product names routed to `substitution`. + The trap parenthetical and the literal-`match` endorsement are gone. +- **A `%s` table**, per extension point, with the measured substitution behavior quoted. +- **A new step 3, "tokens and swap keys are patterns, not literals"**: Go RE2, no + lookaround, implicit word boundaries, live metacharacters, first-wins on overlap, the + measured `ignorecase` behavior, `raw`/`nonword`, and the markdown scoping that spares + URLs and code spans. This one step carries findings c, l, n and C1's neighbours. +- **Step 6 says to read `results[].ruleId`** and states plainly that `success` and the + exit code answer a different question — with the note that exit 1 on `fail/` is correct + for a `level: error` rule and exit 0 is correct for a `warning` one. +- **`check ` is described as not scoped to the rule under test.** +- **`MinAlertLevel` joins the debug ladder**, and the ladder gains a `pass/`-fires branch. +- **The `pass/` bucket is now specified as near-misses**, not correct prose. +- **The three names that must agree** are stated once, together, with the fact that a + mismatch is silent. +- **Claims of CLI enforcement are removed.** "Nothing checks that you wrote all three" is + the honest statement of today's behavior, and step 7 says a whole-project `check` will + not report the fixtures because `.taskless/` is excluded from the walk. + +## Round 2 — in flight + +Re-run against the revised recipe with fresh agents and intents none of round 1 used: + +| Run | Intent | Extension point | Status | +| --- | ------ | --------------- | ------ | +| D | flag "click here" / "read more" as link text | `existence` | pending | +| E | headings in sentence case, with exceptions | `capitalization` (its actual use case) | pending | + +E is the one that matters: it exercises the check whose guidance was wrong in round 1, on +the use case it can actually serve. + +## Planned revision (applied — kept for the record) Batched until B and C report, so the recipe is revised once against all three rather than three times against one. diff --git a/packages/cli/src/help/create-vale-rule.txt b/packages/cli/src/help/create-vale-rule.txt index 38e8fc6..1baeaf8 100644 --- a/packages/cli/src/help/create-vale-rule.txt +++ b/packages/cli/src/help/create-vale-rule.txt @@ -19,7 +19,7 @@ quiet on the prose it should not, scoped to the files it is about. A Vale rule is three files, not one. It is **incomplete until all three exist**, and the failure mode of skipping one is silence rather than an -error — the rule verifies, the check runs, and nothing is ever reported. +error — the rule runs, the check passes, and nothing is ever reported. | # | Artifact | Path | |---|-----------|-------------------------------------------------------------| @@ -27,42 +27,77 @@ error — the rule verifies, the check runs, and nothing is ever reported. | 2 | The scope | a section in `.taskless/vale/.vale.ini` enabling `rules.`| | 3 | Fixtures | `.taskless/vale/rule-tests//pass/` and `.../fail/` | +Nothing checks that you wrote all three. "Required" here means required +to have a working rule, not required by a command that will stop you. + +**Three names must agree**, and nothing reports it when they don't: + +``` +.taskless/vale/rules/no-simply.yml <- the filename +.taskless/vale/rule-tests/no-simply/ <- the fixture directory +rules.no-simply = YES <- the .vale.ini assignment +``` + +A mismatch is not an error. The rule simply never runs. + ## Steps 1. **Say what the rule reads, then pick an extension point.** Write one sentence: "this fires when a document contains ___." Vale rules are - built by extending one of its checks, and the sentence tells you - which: + built by extending one of its eleven checks, and the sentence tells + you which: | If the rule is about… | extends | |-------------------------------------------------------|------------------| | words or phrases that should not appear | `existence` | - | preferring one term over another | `substitution` | - | how something is capitalized (headings, product names)| `capitalization` | + | preferring one term over another — **including the correct spelling of a product name** | `substitution` | + | the case of a whole heading or sentence | `capitalization` | | how many times something may appear | `occurrence` | | a word repeated back to back | `repetition` | | picking one of two acceptable spellings, consistently | `consistency` | | "if X appears, Y must also appear" | `conditional` | | readability or length thresholds | `metric` | - - There are eleven in total. When none of the above fits, read - https://docs.vale.sh/styles for `spelling`, `sequence`, and `script` - before inventing something — Vale has no facility for a rule that - does not extend one of these. + | a misspelling, against a dictionary | `spelling` | + | phrases that must appear in a fixed order | `sequence` | + | anything the above cannot express (Tengo script) | `script` | + + **`capitalization` is about a whole scope, not a word.** It asks + whether an entire heading or sentence matches a case pattern. It + cannot express "the word GitHub, wherever it appears, is spelled + thus" — that is a `substitution`, because you are swapping a wrong + spelling for a right one. Reaching for `capitalization` on a product + name produces a rule that flags whole sentences: measured, a rule with + `match: GitHub` reports `We host on Github and it is fine. should be + GitHub`. + + For the two this recipe does not describe, read + https://docs.vale.sh/styles before inventing something — Vale has no + facility for a rule that does not extend one of these eleven. 2. **Write the style file** to `.taskless/vale/rules/.yml`, where `` is kebab-case and matches the filename. Every rule carries: | Field | Required | Notes | |-----------|----------|---------------------------------------------------------| - | `extends` | yes | one of the extension points above | - | `message` | yes | shown to the user; `%%s` interpolates the match | + | `extends` | yes | one of the eleven above | + | `message` | yes | shown to the user; see the `%%s` table below | | `level` | no | `suggestion` (default), `warning`, or `error` | | `scope` | no | narrow to part of a document, e.g. `heading`, `paragraph`| | `link` | no | a URL the reader can follow for the reasoning | - Then the fields the chosen extension point adds. The three most - common: + **What `%%s` fills with depends on the extension point.** Getting this + wrong is the one mistake in this recipe that passes every check below + — the rule fires, the fixtures are green, and only a human reading the + message sees that it is nonsense. + + | extends | `%%s` count | fills with, left to right | + |----------------|------------|---------------------------| + | `existence` | one | the matched text | + | `substitution` | **two** | the **replacement**, then the matched text | + + Measured: a `substitution` message with a single `%%s` interpolates the + *replacement*, not the match, so `"Use GitHub not %%s"` against the text + `Github` renders `Use GitHub not GitHub`. ```yaml # existence — flag these tokens wherever they appear @@ -76,7 +111,7 @@ error — the rule verifies, the check runs, and nothing is ever reported. ``` ```yaml - # substitution — flag the observed term, name the expected one + # substitution — first %%s is the replacement, second is what was found extends: substitution message: "Use '%%s' instead of '%%s'" level: warning @@ -87,7 +122,7 @@ error — the rule verifies, the check runs, and nothing is ever reported. ``` ```yaml - # capitalization — enforce a case pattern on a part of the document + # capitalization — a whole heading must be in sentence case extends: capitalization message: "'%%s' should be in sentence case" level: warning @@ -98,10 +133,36 @@ error — the rule verifies, the check runs, and nothing is ever reported. - API ``` - `match` takes `$sentence`, `$title`, `$lower`, `$upper`, or a literal - string to enforce verbatim (`Taskless`, for a product name). - -3. **Scope the rule by adding a section to `.taskless/vale/.vale.ini`.** + `match` takes `$sentence`, `$title`, `$lower`, or `$upper`. A literal + string is legal but means "this whole scope must read exactly that", + which is almost never what anyone wants — see step 1. + +3. **Know what you are writing: `tokens` and `swap` keys are patterns, + not literals.** They compile as **Go RE2** regular expressions. + + - `(?:…)`, `[…]`, `|`, `+`, `?` all work. + - **Lookahead and lookbehind do not exist in RE2.** A rule that needs + "X but not when followed by Y" cannot be written as a single + `substitution`; split it or narrow with `scope`. + - **Word boundaries are applied for you.** Measured: the key `Github` + does not fire inside `GithubToken`. + - **Regex metacharacters in a real phrase are live.** "maybe?" is a + pattern meaning "mayb" followed by an optional "e". Escape it. + - **Overlapping alternatives resolve first-wins**, one finding per + match. If `can login` and `login with` both match a sentence, you + get whichever is written first, once — not both. + - `ignorecase: true` matches any casing **and still skips text that + already equals the replacement.** Measured with `Github: GitHub`: + `github` and `Github` are flagged, `GitHub` is not. You do not need + `ignorecase: false` to protect the correct spelling. + - `raw` takes a full regex when `tokens` is too restrictive; + `nonword` removes the implicit boundaries. + + Vale also understands the markup. In markdown, URLs and code spans are + not prose: measured, a `Github` key fires on `Plain Github here` and + not on `https://Github.com/x` or `` `Github/docs` ``. + +4. **Scope the rule by adding a section to `.taskless/vale/.vale.ini`.** This is the step that is easy to skip and impossible to notice skipping. @@ -119,19 +180,20 @@ error — the rule verifies, the check runs, and nothing is ever reported. rules.no-simply = YES ``` - Three things about that block: + - `StylesPath = .` resolves **relative to the directory holding + `.vale.ini`**, which is `.taskless/vale/`. That makes the `rules/` + directory beside it a style named `rules`, which is where the + `rules.` prefix comes from. - `[*.md]` is a glob over paths. Match it to the files the rule is actually about — `[*.{md,mdx}]`, `[docs/**/*.md]`. Every file the - walk reaches that matches the glob is subject to every rule - enabled under it. - - `BasedOnStyles =` left empty says explicitly that no bundled style - loads, so the only findings are the rules you enabled. Write it - even though the default is already empty: it is what stops a later - edit from switching a whole style on by accident, and it is the - form `taskless check` uses when it verifies a rule in isolation. - - `rules. = YES` enables the rule. `rules` is the style name: - `StylesPath = .` makes the `rules/` directory a style, so a file at - `vale/rules/no-simply.yml` is the check `rules.no-simply`. + walk reaches that matches the glob is subject to every rule enabled + under it. + - `BasedOnStyles =` with an empty value is valid and says explicitly + that no bundled style loads. Write it: the default is already empty, + but it is what stops a later edit from switching a whole style on by + accident. + - `MinAlertLevel` is a floor. A rule whose `level` sits below it is + enabled and silent. If a section already covers the right files, add the rule to it rather than opening a second one — a later section matching the same @@ -144,24 +206,27 @@ error — the rule verifies, the check runs, and nothing is ever reported. diagnostic as a notice — if you see it, the assignment is in the wrong place, not the rule. -4. **Write the fixtures.** Two directories, both flat — Vale lints the - rule's whole tree, so a nested subdirectory is linted but never - checked, and the CLI rejects it rather than skipping it: +5. **Write the fixtures.** Two directories, both flat. Vale lints the + rule's whole tree, so a fixture in a nested subdirectory is linted but + never counted against either bucket: ``` .taskless/vale/rule-tests//pass/ok.md # rule must stay quiet .taskless/vale/rule-tests//fail/bad.md # rule must fire ``` - Both buckets are required. A rule with only `fail/` fixtures has - never been shown to stay quiet, which is the half that catches an - over-broad pattern. - Give the fixtures an extension your section's glob matches. A `.txt` fixture under a `[*.md]` section is never linted, so the `fail/` document silently passes. -5. **Check that it fires, and that it doesn't.** Run the rule against + **The `pass/` bucket is not "correct prose".** Correct prose proves + nothing — the rule was never going to fire on it. Fill it with the + near-misses that would catch an over-broad pattern: the noun form you + are not flagging, the word inside a longer word, the term in a URL or + a code span, the correct spelling itself. That is the half of the + fixture set that has to work for you. + +6. **Check that it fires, and that it doesn't.** Run the rule against each bucket: ``` @@ -169,18 +234,39 @@ error — the rule verifies, the check runs, and nothing is ever reported. npx @taskless/cli check .taskless/vale/rule-tests//pass --json ``` - The `fail/` run must report your rule; the `pass/` run must report - nothing. If `fail/` reports nothing, work down the list before - touching the pattern — the cause is usually further up: - - Is the assignment inside a section? (step 3) - - Does the section's glob match the fixture's extension? (step 4) - - Does `rules.` match the filename under `vale/rules/`? + **Read `results`, not `success` and not the exit code.** `check` lints + the path you gave it against the whole config — it is not scoped to + the rule you are testing — so what you are looking for is entries in + `results` whose `ruleId` is yours: + + - `fail/` → at least one `results` entry with `"ruleId": ""`. + - `pass/` → no `results` entry with `"ruleId": ""`. + + `success` is about whether the run worked, not whether your fixture + behaved: a clean `fail/` run reports `"success": true` alongside the + findings you wanted. The exit code follows severity, so a `level: + error` rule exits 1 on `fail/` and a `level: warning` rule exits 0 — + both correct. Neither field answers the question you are asking. + + If `fail/` reports nothing, work down this list before touching the + pattern — the cause is usually further up: + - Is the assignment inside a section? + - Does the section's glob match the fixture's extension? + - Is the rule's `level` at or above `MinAlertLevel`? + - Do the filename, the `rules.` assignment, and the fixture + directory all agree? - Only then: does the pattern actually match the text? -6. **Report.** Show the three paths you wrote, a one-line summary of + If `pass/` reports your rule, the pattern is too broad. Look for a + missing word boundary, an unescaped metacharacter, or a swap key that + also matches the form you meant to allow. + +7. **Report.** Show the three paths you wrote, a one-line summary of what the rule flags, and which files the section scopes it to — the scope is a decision the user should see, not one buried in a config. - Suggest `taskless agent check` to run it over the whole project. + Note that a whole-project `taskless check` will **not** report your + fixtures: `.taskless/` is excluded from the project walk, by design. + The fixtures are exercised by pointing `check` at them, as in step 6. ## Important Notes From 45ba3e621c3158b4c9934bfd254da24c06adb198 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 15:59:41 -0700 Subject: [PATCH 14/19] fix(cli): close the round-2 gaps in create-vale-rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both round-2 agents converged first try, including `capitalization` on the use case round 1 got wrong — so 2b.6 is met across three extension points. They still found nine gaps, three of them in prose written during round 1's revision. The biggest: the field table listed the five fields every rule shares and omitted every field a rule actually needs. `tokens`, `swap`, `match` and `exceptions` appeared only inside examples, so the one field run E's rule depended on was undocumented. There is now a per-extension-point field table. `scope` was likewise an "e.g." with two values for the field that decides where a rule looks; all sixteen markdown scopes are listed. Measured and written down rather than guessed: `match: $sentence` means first word capitalized and everything else lowercase, proper nouns included, so `exceptions` is load-bearing — `Getting started with Kubernetes` fires unless Kubernetes is listed, while an exception's plural and a leading exception are both fine. Link text is prose and the URL is not, so `scope: link` narrows to it. Word boundaries wrap the whole pattern, multi-word included. Two defects were my own from the last pass. A nested-fixture warning described "counting against either bucket", machinery no reader has been shown and a leftover from assuming verifyValeRule was reachable. And the `BasedOnStyles =` rationale claimed it stops a later edit from enabling a style, which is not a mechanism — replaced with the honest reason. 572 tests pass. --- .../iteration-log.md | 70 +++++++++-- .../agent-command-and-vale-authoring/tasks.md | 17 +-- packages/cli/src/help/create-vale-rule.txt | 116 ++++++++++++++---- 3 files changed, 164 insertions(+), 39 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md index 88e277c..66e7d17 100644 --- a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md +++ b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md @@ -183,17 +183,71 @@ changed, beyond the wording items: the honest statement of today's behavior, and step 7 says a whole-project `check` will not report the fixtures because `.taskless/` is excluded from the walk. -## Round 2 — in flight +## Round 2 -Re-run against the revised recipe with fresh agents and intents none of round 1 used: +Re-run against the revised recipe with fresh agents and intents round 1 never used. -| Run | Intent | Extension point | Status | -| --- | ------ | --------------- | ------ | -| D | flag "click here" / "read more" as link text | `existence` | pending | -| E | headings in sentence case, with exceptions | `capitalization` (its actual use case) | pending | +| Run | Intent | Extension point | Converged | +| --- | ------ | --------------- | --------- | +| D | flag "click here" / "read more" as link text | `existence` + `scope: link` | **yes, first try, zero retries** | +| E | headings in sentence case, with exceptions | `capitalization` (its actual use case) | **yes, first try, zero retries** | + +**2b.6 is met.** Three extension points, five runs, and the only failure in the set was +round 1's product-name case — whose cause was corrected and whose check (`capitalization`) +now converges first try on the use case it can actually serve. Both round-2 agents +independently reported that step 6's "read `results`, not `success`" saved them from +misreading a clean `fail/` run, which is the round-1 fix working. -E is the one that matters: it exercises the check whose guidance was wrong in round 1, on -the use case it can actually serve. +Round 2 still found nine gaps, three of them defects in prose written *during* round 1's +revision. Measured before acting on them, as before. + +| # | Finding | Verdict | +| - | ------- | ------- | +| r | The field table lists five common fields under "Every rule carries" and **omits every field a rule actually needs** — `tokens`, `swap`, `match`, `exceptions`. Run E's rule depended entirely on `exceptions`, which appeared only as an undocumented line in an example | **Real, and the round's biggest gap.** Fixed with a per-extension-point field table | +| s | `scope` was documented as "e.g. `heading`, `paragraph`" — an example, not a list — for the field that decides where a rule looks. Run D's rule rested on `scope: link` existing; it guessed | **Real.** All sixteen markdown scopes now listed | +| t | `match: $sentence` was never defined. "The difference between 'first letter capitalized, everything else lowercase' and 'proper nouns permitted' decides whether `Getting started with Kubernetes` fires" | **Real, and the answer matters.** Measured below | +| u | Is `[click here](url)` prose? The recipe answered the inverse — what Vale *excludes* — and never said what link text is | **Real.** Measured below | +| v | Word boundaries were stated for a single-word key only; multi-word behaviour left to assumption | **Real.** Measured: whole-phrase. `click here` does not fire inside `Clicking here` | +| w | The `%%s` table covers 2 of 11 extension points while being billed as the authority on "the one mistake that passes every check" | **Real.** `capitalization` added (measured); the other eight now carry an explicit "don't guess, read it back off the finding" | +| x | Step 3 (RE2, boundaries, `ignorecase`) is irrelevant to `capitalization`/`occurrence`/`metric` and had no skip marker, so run E read all of it looking for `exceptions` semantics | **Real.** Skip line added | +| y | The pass-fixture advice was written for token rules only — "the word inside a longer word" is not a near-miss for a whole-scope check | **Real.** Now branches by rule shape, including "the same phrase *outside* the scope", which is the only thing that proves a `scope` works | +| z | "a fixture in a nested subdirectory is linted but **never counted against either bucket**" references counting machinery the reader has never been shown — a leftover from when `verifyValeRule` was assumed reachable | **Real, my error.** Rewritten to describe what `check` actually does | +| aa | The `BasedOnStyles =` rationale — "it stops a later edit from switching a whole style on by accident" — is not a real mechanism | **Real, my error.** Run D is right: an empty assignment prevents nothing. Replaced with the honest reason (it makes intent readable without knowing the default) | + +Run D repeated run B's objection to the absolute path in step 5. Same answer: `build:dev` +artifact, already recorded as rejected. + +### Measured for round 2 + +**`$sentence` is stricter than "sentence case".** First word capitalized, everything else +lowercase — proper nouns included, unless listed in `exceptions`: + +| Heading | Result | +| ------- | ------ | +| `Getting started with the API` | quiet | +| `Getting started with APIs` | quiet — an exception covers its plural | +| `Taskless and the API` | quiet — an exception may lead the scope | +| `Getting started with Kubernetes` | **fires** — a proper noun not listed | +| `getting started lowercase` | **fires** — the first word must be capitalized | + +So `exceptions` is load-bearing: every proper noun the docs use must be listed or the rule +flags correct headings. That is now in the recipe as a table, and it answers the question +run E said it could not resolve (`APIs`, the plural of an exception, is covered). + +**Link text is prose; the URL is not.** With no `scope`, a `click here` token fired on both +`[click here](https://example.com/x)` and the same phrase in an ordinary sentence. With +`scope: link`, only the link. Both facts are now stated. + +## Round 2 revision applied + +`create-vale-rule.txt`, 286 → 356 lines. Per-extension-point field table; full `scope` +list; `$sentence` semantics as a measured table; link-text scoping; whole-phrase +boundaries; `capitalization` in the `%%s` table with honest guidance for the rest; a skip +marker on step 3; pass-fixture advice branched by rule shape; and the two sentences of my +own that run D correctly called out as describing machinery and mechanisms that do not +exist. + +## Round 1 planned revision (applied — kept for the record) ## Planned revision (applied — kept for the record) diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 96e1e90..28d6b26 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -24,19 +24,20 @@ - [x] 2.6 Rewrite `help/route.txt` to read login state early and classify into the five `create-*-rule` destinations, applying the engine reasoning inline rather than deferring to a second fetch. Offer `create-remote-rule` only where it is a genuine choice — locally expressible AND logged in. A logged-in runtime request routes straight to `create-remote-rule`; a logged-out one to `create-runtime-rule`. It must name a command the agent can run verbatim - [x] 2.7 Merge `help/engine-selection.txt` into `help/route.txt` and delete it. Its three engine definitions, evidence-before-answer procedure, and boundary cases move into `route`'s destination table — stated once, not copied into the destinations - [x] 2.7a Give every `create-*-rule` recipe the same opening orientation line: what topic this is, what it helps you write, and revisit routing if that is not what you need. Fixed shape across all five so an agent recognises it; scope only, never the comparison between engines -- [ ] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic +- [x] 2.8 Re-home the engine-reasoning requirements that survive the merge — "Available code context outranks the phrasing of the request" and "Ambiguity resolves to an engine known to be available" now bind `route` and the destinations. Update `help-extensions.test.ts`, which asserts against the standalone topic ## 2b. Prove the authoring recipes by executing them The recipes are the deliverable, and a recipe that reads well to its author while producing the wrong artifact is exactly what reviewing the prose cannot catch. Execute them instead. -- [ ] 2b.1 `pnpm --filter @taskless/cli build:dev`. This target exists for this: `TASKLESS_BUILD_TARGET=dev` bakes `__TASKLESS_CLI__` as an **absolute path** to `dist-dev/index.js`, so recipe text carries a command that actually runs from any directory. Testing against `dist/` instead would exercise a recipe no reader ever receives, since theirs says `npx @taskless/cli` -- [ ] 2b.2 Build the harness: scaffold a throwaway project in a temp directory using the built CLI, so the sandbox is a real `taskless init` scaffold — section-less `.vale.ini`, empty `vale/rules/` — and not a hand-made approximation of one -- [ ] 2b.3 Hand a **fresh, non-forked** subagent only three things: the recipe text, the sandbox path, and a rule intent stated in plain words. It must NOT have repository access. With it, the agent finds the existing `no-simply.yml` and the mixed-engine fixture and copies them, and the loop tests our fixtures rather than our writing -- [ ] 2b.4 Check the artifacts mechanically: a style file at `.taskless/vale/rules/.yml` with valid `extends`/`message`/`level`; a **scoped section** in `.vale.ini` enabling `rules.`; fixtures in both `pass/` and `fail/`. Then the assertion that matters — `check` reports the finding, and `verify` passes -- [ ] 2b.5 Iterate across three intents that exercise different extension points — one `existence`, one `substitution` (prefer X over Y), one `capitalization` (headings, product names). Everything in this repo today is `existence`, so a recipe drafted from our own examples teaches token blocklists and nothing else. Vale has eleven extension points and most real prose rules are not blocklists -- [ ] 2b.6 Every failure is a defect in the prose, not in the agent. Fix the recipe and re-run with a fresh agent. Converged when an agent, given an intent the recipe never names, produces a rule that fires on its `fail` fixture and stays quiet on its `pass` fixture, first try, uncorrected -- [ ] 2b.7 Keep the iteration log — what failed, what changed, what finally held. It is the evidence the prose works, and the only part of this a reviewer can check without rerunning the loop +- [x] 2b.1 `pnpm --filter @taskless/cli build:dev`. This target exists for this: `TASKLESS_BUILD_TARGET=dev` bakes `__TASKLESS_CLI__` as an **absolute path** to `dist-dev/index.js`, so recipe text carries a command that actually runs from any directory. Testing against `dist/` instead would exercise a recipe no reader ever receives, since theirs says `npx @taskless/cli` +- [x] 2b.2 Build the harness: scaffold a throwaway project in a temp directory using the built CLI, so the sandbox is a real `taskless init` scaffold — section-less `.vale.ini`, empty `vale/rules/` — and not a hand-made approximation of one +- [x] 2b.3 Hand a **fresh, non-forked** subagent only three things: the recipe text, the sandbox path, and a rule intent stated in plain words. It must NOT have repository access. With it, the agent finds the existing `no-simply.yml` and the mixed-engine fixture and copies them, and the loop tests our fixtures rather than our writing +- [x] 2b.4 Check the artifacts mechanically: a style file at `.taskless/vale/rules/.yml` with valid `extends`/`message`/`level`; a **scoped section** in `.vale.ini` enabling `rules.`; fixtures in both `pass/` and `fail/`. Then the assertion that matters — `check` reports the finding, and `verify` passes + - **Done except "`verify` passes", which cannot be satisfied as written.** `verifyValeRule`/`verifyValeRules` have no CLI caller: `taskless rule verify ` routes to `src/rules/verify.ts`, which is ast-grep only. There is no way for an agent to verify a Vale rule from the CLI today. The harness asserts on `check` over each bucket instead, which is what `create-vale-rule` now tells authors to do. **Open decision for the user** — wire `rule verify ` to dispatch by owning engine, or file it alongside #99/#101. Until then the recipe says plainly that nothing validates the fixture layout, rather than claiming a check that does not run +- [x] 2b.5 Iterate across three intents that exercise different extension points — one `existence`, one `substitution` (prefer X over Y), one `capitalization` (headings, product names). Everything in this repo today is `existence`, so a recipe drafted from our own examples teaches token blocklists and nothing else. Vale has eleven extension points and most real prose rules are not blocklists +- [x] 2b.6 Every failure is a defect in the prose, not in the agent. Fix the recipe and re-run with a fresh agent. Converged when an agent, given an intent the recipe never names, produces a rule that fires on its `fail` fixture and stays quiet on its `pass` fixture, first try, uncorrected +- [x] 2b.7 Keep the iteration log — what failed, what changed, what finally held. It is the evidence the prose works, and the only part of this a reviewer can check without rerunning the loop - [ ] 2b.8 Run the same harness over `create-sg-rule` as a control. It documents a flow that already works, so a failure there means the harness is wrong rather than the recipe ## 3. Sweep the cross-references diff --git a/packages/cli/src/help/create-vale-rule.txt b/packages/cli/src/help/create-vale-rule.txt index 1baeaf8..cd04e69 100644 --- a/packages/cli/src/help/create-vale-rule.txt +++ b/packages/cli/src/help/create-vale-rule.txt @@ -82,23 +82,49 @@ A mismatch is not an error. The rule simply never runs. | `extends` | yes | one of the eleven above | | `message` | yes | shown to the user; see the `%%s` table below | | `level` | no | `suggestion` (default), `warning`, or `error` | - | `scope` | no | narrow to part of a document, e.g. `heading`, `paragraph`| + | `scope` | no | narrow to part of a document — see below | | `link` | no | a URL the reader can follow for the reasoning | + **`scope` decides where the rule looks**, so getting it wrong is a + silent over- or under-fire rather than an error. The useful values for + markdown: `heading` (and `heading.h1`…`heading.h6`), `paragraph`, + `sentence`, `list`, `blockquote`, `link`, `code`, `alt`, `summary`, + `table.header`, `table.cell`, `table.caption`, `figure.caption`, + `strong`, `emphasis`, `raw`. Omit it and the rule sees all prose. + + **Then the fields the extension point adds** — this is where the rule + actually lives, and each check reads only its own: + + | extends | its fields | + |------------------|-------------------------------------------------------------------| + | `existence` | `tokens` (a list) or `raw`; `ignorecase`, `nonword`, `exceptions`, `append` | + | `substitution` | `swap` (a map of observed → expected); `ignorecase`, `nonword`, `exceptions` | + | `capitalization` | `match`; `exceptions`, `style` (with `$title`), `threshold`, `indicators`, `prefix` | + | `occurrence` | `token`, `max` and/or `min` | + | `repetition` | `tokens`; `alpha`, `ignorecase` | + | `consistency` | `either` (a map of the two acceptable forms) | + | `conditional` | `first`, `second`; `exceptions` | + | `metric` | `formula`, `condition` | + **What `%%s` fills with depends on the extension point.** Getting this wrong is the one mistake in this recipe that passes every check below — the rule fires, the fixtures are green, and only a human reading the message sees that it is nonsense. - | extends | `%%s` count | fills with, left to right | - |----------------|------------|---------------------------| - | `existence` | one | the matched text | - | `substitution` | **two** | the **replacement**, then the matched text | + | extends | `%%s` count | fills with, left to right | + |------------------|------------|---------------------------| + | `existence` | one | the matched text | + | `substitution` | **two** | the **replacement**, then the matched text | + | `capitalization` | one | the scope that failed (the whole heading or sentence) | Measured: a `substitution` message with a single `%%s` interpolates the *replacement*, not the match, so `"Use GitHub not %%s"` against the text `Github` renders `Use GitHub not GitHub`. + For the other eight, do not guess. Write the message, run step 6, and + read it back off the finding — no test you can write catches a wrong + `%%s`, so your own eyes on the rendered message are the check. + ```yaml # existence — flag these tokens wherever they appear extends: existence @@ -137,15 +163,37 @@ A mismatch is not an error. The rule simply never runs. string is legal but means "this whole scope must read exactly that", which is almost never what anyone wants — see step 1. + **`$sentence` means first word capitalized, everything else lowercase + — proper nouns included.** It is not "sentence case allowing proper + nouns". Measured with `exceptions: [Taskless, API]` on headings: + + | Heading | Result | + |------------------------------------|--------| + | `Getting started with the API` | quiet | + | `Getting started with APIs` | quiet — an exception covers its plural | + | `Taskless and the API` | quiet — an exception may lead the scope | + | `Getting started with Kubernetes` | **fires** — a proper noun you did not list | + | `getting started lowercase` | **fires** — the first word must be capitalized | + | `Getting Started With Title Case` | **fires** | + + So `exceptions` is not decoration: every proper noun, product name and + acronym the docs use has to be listed, or the rule flags correct + headings. Collect them from the docs before writing the rule, and + expect to add to the list. + 3. **Know what you are writing: `tokens` and `swap` keys are patterns, not literals.** They compile as **Go RE2** regular expressions. + *This step is about `tokens` and `swap` only. A `capitalization`, + `occurrence` or `metric` rule has neither — skip to step 4.* + - `(?:…)`, `[…]`, `|`, `+`, `?` all work. - **Lookahead and lookbehind do not exist in RE2.** A rule that needs "X but not when followed by Y" cannot be written as a single `substitution`; split it or narrow with `scope`. - - **Word boundaries are applied for you.** Measured: the key `Github` - does not fire inside `GithubToken`. + - **Word boundaries are applied for you, around the whole pattern.** + Measured: `Github` does not fire inside `GithubToken`, and the + multi-word `click here` does not fire inside `Clicking here`. - **Regex metacharacters in a real phrase are live.** "maybe?" is a pattern meaning "mayb" followed by an optional "e". Escape it. - **Overlapping alternatives resolve first-wins**, one finding per @@ -158,9 +206,17 @@ A mismatch is not an error. The rule simply never runs. - `raw` takes a full regex when `tokens` is too restrictive; `nonword` removes the implicit boundaries. - Vale also understands the markup. In markdown, URLs and code spans are - not prose: measured, a `Github` key fires on `Plain Github here` and - not on `https://Github.com/x` or `` `Github/docs` ``. + Vale also understands the markup, which decides what counts as text + before your pattern ever runs. Measured in markdown: + + - **URLs and code spans are not prose.** A `Github` key fires on + `Plain Github here` and not on `https://Github.com/x` or + `` `Github/docs` ``. + - **Link text *is* prose.** In `[click here](https://example.com)`, + `click here` is matched — the URL is not. With no `scope`, a rule + fires on both link text and ordinary prose; `scope: link` narrows it + to link text alone. Measured: without a scope the token hit both the + link and the sentence; with `scope: link`, only the link. 4. **Scope the rule by adding a section to `.taskless/vale/.vale.ini`.** This is the step that is easy to skip and impossible to notice @@ -189,9 +245,10 @@ A mismatch is not an error. The rule simply never runs. walk reaches that matches the glob is subject to every rule enabled under it. - `BasedOnStyles =` with an empty value is valid and says explicitly - that no bundled style loads. Write it: the default is already empty, - but it is what stops a later edit from switching a whole style on by - accident. + that no bundled style loads. The default is already empty, so this + changes nothing today; write it because it makes the section's + intent readable to the next person, who would otherwise have to know + the default to know whether a style is missing or deliberately off. - `MinAlertLevel` is a floor. A rule whose `level` sits below it is enabled and silent. @@ -206,9 +263,11 @@ A mismatch is not an error. The rule simply never runs. diagnostic as a notice — if you see it, the assignment is in the wrong place, not the rule. -5. **Write the fixtures.** Two directories, both flat. Vale lints the - rule's whole tree, so a fixture in a nested subdirectory is linted but - never counted against either bucket: +5. **Write the fixtures.** Two directories, both flat. Step 6 points + `check` at one bucket at a time and lints everything beneath it, so a + fixture in a nested subdirectory still gets linted — its findings just + arrive mixed into whichever bucket you were testing, where they prove + nothing. Keep both buckets one level deep: ``` .taskless/vale/rule-tests//pass/ok.md # rule must stay quiet @@ -221,13 +280,24 @@ A mismatch is not an error. The rule simply never runs. **The `pass/` bucket is not "correct prose".** Correct prose proves nothing — the rule was never going to fire on it. Fill it with the - near-misses that would catch an over-broad pattern: the noun form you - are not flagging, the word inside a longer word, the term in a URL or - a code span, the correct spelling itself. That is the half of the - fixture set that has to work for you. - -6. **Check that it fires, and that it doesn't.** Run the rule against - each bucket: + near-misses that would catch an over-broad pattern. What counts as a + near-miss depends on the rule's shape: + + - **`tokens`/`swap` rules** — the noun form you are not flagging, the + word inside a longer word, the term in a URL or a code span, the + correct spelling itself. + - **`scope`d rules** — the same phrase *outside* the scope. A rule + with `scope: link` needs the phrase in ordinary prose; a rule with + `scope: heading` needs it in body text. Without that, nothing proves + the scope is doing anything. + - **`capitalization` rules** — a scope that is entirely exceptions, a + scope whose exception word comes first, and the plural of an + exception. + + That is the half of the fixture set that has to work for you. + +6. **Check that it fires, and that it doesn't.** Run each bucket, from + the project root — the paths below are relative to it: ``` npx @taskless/cli check .taskless/vale/rule-tests//fail --json From 58c5a55451bdda380fe7cb1910ad20a45199343e Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 15:59:59 -0700 Subject: [PATCH 15/19] docs(openspec): record 2b as complete but for its control run --- .../resume.md | 26 ++++++++++--------- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/openspec/changes/agent-command-and-vale-authoring/resume.md b/openspec/changes/agent-command-and-vale-authoring/resume.md index db8ac97..e392002 100644 --- a/openspec/changes/agent-command-and-vale-authoring/resume.md +++ b/openspec/changes/agent-command-and-vale-authoring/resume.md @@ -75,18 +75,20 @@ have exercised a recipe nobody will receive. ## Next up: finish 2b (the harness), then 5.3/5.4, then group 3 -### 2b is mid-flight — read `iteration-log.md` first - -**Run A has reported and is written up in `iteration-log.md`**, with its findings triaged -against the source and a nine-item planned revision. It converged first try, which is -worth reading skeptically: the step-5 debug ladder went untested, and the findings are -mostly about rungs that ladder is missing. **Item 3 of the planned revision needs a -decision, not a wording pass** — the recipe currently asserts fixture validation that no -CLI command performs (see "A real gap" below). - -Runs B (`substitution`) and C (`capitalization`) had **not reported** when the session -ended. The revision is deliberately batched until all three are in, so the recipe is -revised once against three reports rather than three times against one. Their sandboxes are under +### 2b is done except its control run — read `iteration-log.md` for the evidence + +Five sandboxed runs across two rounds, three extension points, twenty-five findings. +**2b.1–2b.7 are complete and 2b.6 is met**: an agent given an intent the recipe never +names now produces a working rule first try, uncorrected, for `existence`, +`substitution` and `capitalization`. `create-vale-rule` went 200 → 356 lines across two +revisions. + +**Only 2b.8 remains** — run the same harness over `create-sg-rule` as a control. A failure +there means the harness is wrong rather than the recipe. The procedure is below and the +scratchpad sandboxes are machine-local, so re-create them. + +**One task is checked off with a caveat you should read: 2b.4.** "`verify` passes" cannot +be satisfied — see "A real gap" below. It is the one open decision in this group. Their sandboxes are under the scratchpad at `vale-harness/sandbox-{a,b,c}`, each a real `init` scaffold, alongside `create-vale-rule.rendered.txt` (the dev-build render they were given). **That scratchpad is machine-local — on another machine, re-run the harness from scratch rather than looking From 19b36015734906d16ed5124e7567cdf5bbfc666c Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 18:15:59 -0700 Subject: [PATCH 16/19] docs(cli): add nine worked Vale rules, each with its near-miss MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both harness rounds converged, but every run invented its rule shape from three examples, and the recurring note in all five reports was "I guessed and happened to be right". The recipe explained the mechanics well and showed almost nothing. Nine worked rules, one per extension point covered, each paired with the near-miss that fails and why. The YAML blocks were extracted from the *rendered* recipe and executed verbatim, so what ships is what was tested rather than something adjacent to it — all nine fire with the expected message, and the negative cases stay quiet (the correct `GitHub` is not flagged; `scope: link` spares the same phrase in prose; a defined acronym licenses its later uses). Writing them surfaced a failure mode no agent hit: an unquoted `[^\s]+` in a repetition rule's tokens matches nothing — no error, no diagnostic, zero findings. It is a YAML escaping problem that presents exactly like a Vale scoping problem, and it is now example 7's "goes wrong". Three others are behaviors an agent would not guess: `consistency` enforces internal consistency rather than picking a winner, `occurrence` counts per scope, and `conditional`'s first/second invert the rule when swapped. 572 tests pass; lint clean. --- .../iteration-log.md | 36 ++++ packages/cli/src/help/create-vale-rule.txt | 189 ++++++++++++++++++ 2 files changed, 225 insertions(+) diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md index 66e7d17..cfe1308 100644 --- a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md +++ b/openspec/changes/agent-command-and-vale-authoring/iteration-log.md @@ -247,6 +247,42 @@ marker on step 3; pass-fixture advice branched by rule shape; and the two senten own that run D correctly called out as describing machinery and mechanisms that do not exist. +## Round 3 — worked examples + +Both rounds converged, but every run had to *invent* its rule shape from three examples, +and the near-misses each one flagged as "I guessed and happened to be right" were the +recurring theme. The recipe explained the mechanics well and showed almost nothing. + +Added a **Worked rules** section: nine rules, one per extension point the recipe covers, +each paired with the near-miss that fails and why. 356 → 545 lines. + +Every example is measured, and — the part that matters — the **YAML blocks were extracted +from the rendered recipe and executed verbatim**, so what ships is what was tested rather +than something adjacent to it: + +| # | Extension point | Result | +| - | --------------- | ------ | +| 1 | `existence` (hedging) | fires ×3 — `"Avoid hedging: 'We think'"` | +| 2 | `substitution` (login → sign in) | fires ×2 — `"Use 'sign in to' instead of 'login to'"` | +| 3 | `substitution` (GitHub) | fires ×2 on `Github`/`github`, **not** on the correct `GitHub` | +| 4 | `capitalization` (sentence case) | fires ×1 on a Title Case heading | +| 5 | `existence` + `scope: link` | fires ×1 — the link only, not the same phrase in prose | +| 6 | `occurrence` (max 1 per paragraph) | fires ×1 | +| 7 | `repetition` (doubled word) | fires ×1 — `"'is' is repeated"` | +| 8 | `consistency` (-ize/-ise) | fires ×1 | +| 9 | `conditional` (define the acronym) | fires ×1 on the undefined `XYZ`, not the defined `API` | + +**One near-miss was found by writing the examples, not by an agent.** An unquoted +`[^\s]+` in a `repetition` rule's `tokens` matches nothing — zero findings, no error, no +diagnostic. It is a YAML escaping failure that presents exactly like a Vale scoping +failure, and it is now example 7's "goes wrong". Worth noting that this is the failure +mode the recipe warns about most, arriving through a layer the recipe had not covered. + +Three other "goes wrong" entries are behaviors an agent would not guess: `consistency` +enforces internal consistency rather than picking a winner (use `substitution` for house +style); `occurrence` counts per `scope`, so omitting it caps the document; and +`conditional`'s `first`/`second` invert the rule if swapped. + ## Round 1 planned revision (applied — kept for the record) ## Planned revision (applied — kept for the record) diff --git a/packages/cli/src/help/create-vale-rule.txt b/packages/cli/src/help/create-vale-rule.txt index cd04e69..96547df 100644 --- a/packages/cli/src/help/create-vale-rule.txt +++ b/packages/cli/src/help/create-vale-rule.txt @@ -74,6 +74,11 @@ A mismatch is not an error. The rule simply never runs. https://docs.vale.sh/styles before inventing something — Vale has no facility for a rule that does not extend one of these eleven. + **Nine of them have a worked rule at the end of this recipe**, each + with the near-miss that fails and why. Read the one closest to your + intent before writing anything — the mistakes documented there are + observed, and most of them fail silently. + 2. **Write the style file** to `.taskless/vale/rules/.yml`, where `` is kebab-case and matches the filename. Every rule carries: @@ -338,6 +343,190 @@ A mismatch is not an error. The rule simply never runs. fixtures: `.taskless/` is excluded from the project walk, by design. The fixtures are exercised by pointing `check` at them, as in step 6. +## Worked rules + +Eight rules that work, each paired with the near-miss that fails. Every +one was run against the bundled Vale; the "what goes wrong" lines are +observed behavior, not warnings in principle. Find the entry closest to +your intent and start there. + +### 1. Ban a word or phrase — `existence` + +> "Our docs shouldn't hedge." + +```yaml +extends: existence +message: "Avoid hedging: '%%s'" +level: warning +ignorecase: true +tokens: + - we think + - it seems + - sort of +``` + +**Goes wrong:** dropping `ignorecase: true` when you meant any casing — +`We think` at the start of a sentence then sails through. And a phrase +with punctuation is a *pattern*: `maybe?` means "mayb" plus an optional +"e", so it matches `mayb`. Escape it: `maybe\?`. + +### 2. Prefer one term over another — `substitution` + +> "Say 'sign in', not 'login', when it's a verb." + +```yaml +extends: substitution +message: "Use '%%s' instead of '%%s'" +level: warning +ignorecase: true +swap: + 'login (?:to|into)': sign in to + 'to login': to sign in +``` + +**Goes wrong:** one `%%s` instead of two. Measured, `"Use sign in not +%%s"` against `login to` renders **"Use sign in not sign in to"** — the +replacement, twice. The rule fires, both fixtures pass, and only a human +reading the message sees it. Two `%%s`, always, in that order. + +### 3. Enforce a product's spelling — `substitution`, not `capitalization` + +> "It's 'GitHub', never 'Github' or 'github'." + +```yaml +extends: substitution +message: "Use '%%s' instead of '%%s'" +level: error +ignorecase: true +swap: + github: GitHub +``` + +**Goes wrong:** reaching for `capitalization` because the complaint is +about capitals. Measured, `match: GitHub` flags whole sentences — +`'We host on Github and it is fine. should be GitHub'` — because that +check tests a *scope*, not a word. Note also that `ignorecase: true` is +safe here: Vale skips text already equal to the replacement, so the +correct `GitHub` is not flagged. + +### 4. Sentence-case headings — `capitalization` + +> "Headings are sentence case; our product names keep their capitals." + +```yaml +extends: capitalization +message: "'%%s' should be in sentence case" +level: warning +scope: heading +match: $sentence +exceptions: + - Taskless + - API + - Kubernetes +``` + +**Goes wrong:** a short `exceptions` list. `$sentence` lowercases +everything after the first word, proper nouns included, so every product +name and acronym in the docs must be listed or correct headings get +flagged. Collect them from the docs first; expect to add more. + +### 5. Restrict a rule to link text — any check, plus `scope` + +> "'click here' is useless link text." + +```yaml +extends: existence +message: "Link text '%%s' says nothing — name the destination" +level: warning +scope: link +ignorecase: true +tokens: + - click here + - read more +``` + +**Goes wrong:** omitting `scope: link`. Measured, the token then fires +on `[click here](…)` **and** on "click here to focus the search box" in +ordinary prose, which is a false positive on a sentence that is fine. +Whenever a rule is about a *place* in the document, the `pass/` fixture +must contain the same phrase outside that place — otherwise nothing +proves the scope works. + +### 6. Cap how often something appears — `occurrence` + +> "At most one exclamation mark per paragraph." + +```yaml +extends: occurrence +message: "Too many exclamation marks" +level: warning +scope: paragraph +token: "!" +max: 1 +``` + +**Goes wrong:** forgetting `scope`. The count is per scope, so with no +scope you are capping the whole document rather than the paragraph. +Note `token` here is singular — this check takes one, not a `tokens` list. + +### 7. Catch a doubled word — `repetition` + +> "'the the' keeps slipping through review." + +```yaml +extends: repetition +message: "'%%s' is repeated" +level: warning +alpha: true +tokens: + - '[^\s]+' +``` + +**Goes wrong:** leaving the pattern unquoted. Measured, an unquoted +`[^\s]+` in YAML silently matches nothing — zero findings, no error, no +diagnostic. Quote any pattern containing a backslash. This is the +failure mode this recipe warns about most, arriving through YAML rather +than through Vale. + +### 8. One spelling or the other, consistently — `consistency` + +> "Pick -ize or -ise and stick to it." + +```yaml +extends: consistency +message: "Use '%%s' consistently" +level: warning +nonword: true +either: + organize: organise +``` + +**Goes wrong:** expecting it to pick a winner. It flags the *second* +form once both appear in a document — it enforces internal consistency, +not house style. If you want one specific spelling, that is a +`substitution`. + +### 9. Require a definition — `conditional` + +> "An acronym must be spelled out before it's used." + +```yaml +extends: conditional +message: "'%%s' has no definition" +level: warning +scope: text +ignorecase: false +first: '\b([A-Z]{3,5})\b' +second: '(?:\b[A-Z][a-z]+ )+\(([A-Z]{3,5})\)' +``` + +`first` is what must be justified; `second` is what justifies it. +Measured: `Application Programming Interface (API)` licenses every later +`API`, while an undefined `XYZ` is flagged. + +**Goes wrong:** swapping the two, which inverts the rule into "flag the +definition when the acronym is missing". + ## Important Notes - Vale reads one document at a time and has no cross-document view. A From bc09897a909473da64dd52d531eed743e3787b4d Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Thu, 13 Aug 2026 18:36:23 -0700 Subject: [PATCH 17/19] feat(cli): verify Vale rules, enforcing the fixture layout MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `verifyValeRule` has existed since the Vale engine landed and had no CLI caller — `rule verify ` routed to the ast-grep verifier only, so there was no way to verify a Vale rule from the CLI. Every harness run noticed, because the recipe had to describe a layout nothing enforced. Baking it in now rather than later: once people write Vale rules against an unenforced layout, changing it is a migration. `rule verify ` now dispatches on which engine owns the rule, decided by where the file sits — the same rule `dispatch` follows, so a rule cannot be verified by one engine and run by another. An id present under both engines is an error naming both paths rather than a silent pick. The two reports stay separate shapes behind an `engine` discriminant. ast-grep verification is three layers over a rule and its test cases; Vale verification is one question asked of two fixture buckets. Mapping the second onto schema/requirements/tests would invent two empty layers and lose `fixtures`, which is the field that catches a rule that was never really verified — one bucket proves half of what a rule needs. Enforced, measured end to end: both buckets populated or it fails without running Vale; every fail/ document must fire; no pass/ document may; a nested fixture directory is rejected by name rather than skipped. Adds ENGINE_UNAVAILABLE to the error envelope, kept apart from SCAN_FAILED because 'install something' and 'fix something' are different asks. 8 new tests, 580 passing. --- packages/cli/src/commands/rules.ts | 146 ++++++++- packages/cli/src/help/create-vale-rule.txt | 87 ++++-- packages/cli/src/rules/owner.ts | 73 +++++ packages/cli/src/schemas/rules-verify.ts | 36 +++ packages/cli/src/types/errors.ts | 5 + .../cli/test/rule-verify-dispatch.test.ts | 277 ++++++++++++++++++ 6 files changed, 589 insertions(+), 35 deletions(-) create mode 100644 packages/cli/src/rules/owner.ts create mode 100644 packages/cli/test/rule-verify-dispatch.test.ts diff --git a/packages/cli/src/commands/rules.ts b/packages/cli/src/commands/rules.ts index 2033927..b1d8b47 100644 --- a/packages/cli/src/commands/rules.ts +++ b/packages/cli/src/commands/rules.ts @@ -6,6 +6,8 @@ import { ZodError } from "zod"; import { resolveIdentity } from "../auth/identity"; import { verifyRule } from "../rules/verify"; +import { verifyValeRule } from "../rules/vale/verify"; +import { rulefileOwners, ruleFileLocation } from "../rules/owner"; import { submitRule, pollRuleStatus, iterateRule } from "../api/rules"; import { writeRuleFile, @@ -24,7 +26,10 @@ import { outputSchema as improveOutputSchema, } from "../schemas/rules-improve"; import { outputSchema as metaOutputSchema } from "../schemas/rules-meta"; -import { verifyOutputSchema } from "../schemas/rules-verify"; +import { + verifyOutputSchema, + valeVerifyOutputSchema, +} from "../schemas/rules-verify"; import { getTelemetry } from "../telemetry"; import { CLIError } from "../util/cli-error"; import { type CLIErrorCode, makeErrorEnvelope } from "../types/errors"; @@ -683,10 +688,112 @@ const deleteCommand = defineCommand({ }, }); +/** + * Verify a Vale rule against its fixture buckets. + * + * Split out because the two engines answer different questions and their + * reports share no fields beyond `ruleId`/`success`. Vale's failure modes are + * about the *fixtures*: a bucket nobody populated proves nothing, and reporting + * that as a pass is how an unverified rule ships looking verified. + */ +async function verifyValeRuleCommand( + cwd: string, + ruleId: string, + json: boolean +): Promise { + let result; + try { + result = await verifyValeRule(cwd, ruleId); + } catch (error) { + // A nested fixture directory throws rather than being skipped: Vale lints + // the rule's whole tree, so a nested document is linted but never checked + // against a bucket, and silently ignoring it would let half a rule's + // fixtures go unverified while it reported a pass. + const message = error instanceof Error ? error.message : String(error); + if (json) { + console.log(JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message))); + } else { + console.error(`Error: ${message}`); + } + process.exitCode = 1; + return; + } + + if ("outcome" in result) { + // Vale never ran. Not a verification result, and not a pass. + const { outcome } = result; + if (json) { + console.log( + JSON.stringify( + makeErrorEnvelope( + outcome.status === "unavailable" + ? "ENGINE_UNAVAILABLE" + : "SCAN_FAILED", + outcome.message + ) + ) + ); + } else { + console.error(`Error: ${outcome.message}`); + } + process.exitCode = 1; + return; + } + + if (json) { + console.log( + JSON.stringify( + valeVerifyOutputSchema.parse({ + engine: "vale", + success: result.passed, + ruleId: result.ruleId, + fixtures: result.fixtures, + missingFailures: result.missingFailures, + unexpectedFindings: result.unexpectedFindings, + }) + ) + ); + } else { + console.log(`Verifying Vale rule: ${result.ruleId}\n`); + + const coverage: Record = { + both: "✓ pass/ and fail/ both have documents", + "pass-only": "✗ no fail/ fixtures — the rule is never shown to fire", + "fail-only": + "✗ no pass/ fixtures — the rule is never shown to stay quiet", + none: "✗ no fixtures at all", + }; + console.log(`Fixtures: ${coverage[result.fixtures]}`); + + console.log( + `Fires: ${result.missingFailures.length === 0 ? "✓ every fail/ document was flagged" : "✗ some fail/ documents were not flagged"}` + ); + for (const file of result.missingFailures) { + console.log(` - ${file}`); + } + + console.log( + `Stays quiet: ${result.unexpectedFindings.length === 0 ? "✓ no pass/ document was flagged" : "✗ some pass/ documents were flagged"}` + ); + for (const file of result.unexpectedFindings) { + console.log(` - ${file}`); + } + + console.log( + `\nResult: ${result.passed ? "✓ All checks passed" : "✗ Verification failed"}` + ); + } + + if (!result.passed) { + process.exitCode = 1; + } +} + const verifyCommand = defineCommand({ meta: { name: "verify", - description: "Validate a rule against the ast-grep schema and run tests", + description: + "Validate a rule and run its tests (ast-grep) or its fixtures (Vale)", }, args: { dir: { @@ -729,10 +836,41 @@ const verifyCommand = defineCommand({ return; } - const result = await verifyRule(cwd, args.id); + // Which engine owns the rule is decided by where its file sits, the same + // way `dispatch` decides it, so a rule cannot be verified by one engine and + // run by another. + const ruleId = args.id; + const owners = await rulefileOwners(cwd, ruleId); + + if (owners.length > 1) { + // Both engines hold this id. Verifying one silently would report on a + // file the user may not have meant, so name both and let them say which. + const message = + `Rule "${ruleId}" exists for more than one engine: ` + + owners.map((engine) => ruleFileLocation(engine, ruleId)).join(", ") + + ". Rename one so the id identifies a single rule."; + if (args.json) { + console.log( + JSON.stringify(makeErrorEnvelope("INVALID_INPUT", message)) + ); + } else { + console.error(`Error: ${message}`); + } + process.exitCode = 1; + return; + } + + if (owners[0] === "vale") { + await verifyValeRuleCommand(cwd, ruleId, args.json); + return; + } + + // No owner falls through to the ast-grep verifier, which already reports a + // missing rule file across its layers — including the legacy location. + const result = await verifyRule(cwd, ruleId); if (args.json) { - console.log(JSON.stringify(verifyOutputSchema.parse(result))); + console.log(JSON.stringify(verifyOutputSchema.parse({ engine: "sg", ...result }))); } else { console.log(`Verifying rule: ${result.ruleId}\n`); diff --git a/packages/cli/src/help/create-vale-rule.txt b/packages/cli/src/help/create-vale-rule.txt index 96547df..796f794 100644 --- a/packages/cli/src/help/create-vale-rule.txt +++ b/packages/cli/src/help/create-vale-rule.txt @@ -27,10 +27,12 @@ error — the rule runs, the check passes, and nothing is ever reported. | 2 | The scope | a section in `.taskless/vale/.vale.ini` enabling `rules.`| | 3 | Fixtures | `.taskless/vale/rule-tests//pass/` and `.../fail/` | -Nothing checks that you wrote all three. "Required" here means required -to have a working rule, not required by a command that will stop you. +`verify-rule` checks all three, and refuses a rule that has only one +fixture bucket — a rule with no `fail/` was never shown to fire, and one +with no `pass/` was never shown to stay quiet. Either alone is half a +claim, so neither passes. -**Three names must agree**, and nothing reports it when they don't: +**Three names must agree:** ``` .taskless/vale/rules/no-simply.yml <- the filename @@ -38,7 +40,14 @@ to have a working rule, not required by a command that will stop you. rules.no-simply = YES <- the .vale.ini assignment ``` -A mismatch is not an error. The rule simply never runs. +`verify-rule` catches the first two: it builds its own config from the +id you give it, so a misnamed rule file or fixture directory shows up as +fixtures that never fired, or as no fixtures at all. + +**It cannot catch the third.** Verification runs in an isolated config +of its own making, so a rule can verify perfectly and still be absent +from the committed `.vale.ini` that `check` reads — verified, and never +run. That assignment is the one thing you have to get right by looking. ## Steps @@ -268,11 +277,10 @@ A mismatch is not an error. The rule simply never runs. diagnostic as a notice — if you see it, the assignment is in the wrong place, not the rule. -5. **Write the fixtures.** Two directories, both flat. Step 6 points - `check` at one bucket at a time and lints everything beneath it, so a - fixture in a nested subdirectory still gets linted — its findings just - arrive mixed into whichever bucket you were testing, where they prove - nothing. Keep both buckets one level deep: +5. **Write the fixtures.** Two directories, both flat. Vale lints the + rule's whole tree, so a document in a nested subdirectory would be + linted but never checked against either bucket — `verify-rule` rejects + one by name rather than skipping it. Keep both buckets one level deep: ``` .taskless/vale/rule-tests//pass/ok.md # rule must stay quiet @@ -301,30 +309,47 @@ A mismatch is not an error. The rule simply never runs. That is the half of the fixture set that has to work for you. -6. **Check that it fires, and that it doesn't.** Run each bucket, from - the project root — the paths below are relative to it: +6. **Verify it.** One command, run from the project root: ``` - npx @taskless/cli check .taskless/vale/rule-tests//fail --json - npx @taskless/cli check .taskless/vale/rule-tests//pass --json + npx @taskless/cli rule verify --json + ``` + + It runs your rule against both buckets in an isolated config — only + your rule enabled, no bundled styles — and answers the question the + fixtures were written to ask: + + ```json + {"engine":"vale","success":true,"ruleId":"no-simply", + "fixtures":"both","missingFailures":[],"unexpectedFindings":[]} ``` - **Read `results`, not `success` and not the exit code.** `check` lints - the path you gave it against the whole config — it is not scoped to - the rule you are testing — so what you are looking for is entries in - `results` whose `ruleId` is yours: + - `success: true` and `fixtures: "both"` is the only passing shape. + - `fixtures` other than `"both"` fails without running Vale: a rule + with one bucket has proved half of what it needs to. + - `missingFailures` names `fail/` documents that were not flagged. + - `unexpectedFindings` names `pass/` documents that were. - - `fail/` → at least one `results` entry with `"ruleId": ""`. - - `pass/` → no `results` entry with `"ruleId": ""`. + Exit code is 0 on success and 1 on any failure, so this one *is* safe + to script. + + If you would rather see the raw findings — the message text, the line + numbers — run `check` against a bucket instead: + + ``` + npx @taskless/cli check .taskless/vale/rule-tests//fail --json + ``` - `success` is about whether the run worked, not whether your fixture - behaved: a clean `fail/` run reports `"success": true` alongside the - findings you wanted. The exit code follows severity, so a `level: - error` rule exits 1 on `fail/` and a `level: warning` rule exits 0 — - both correct. Neither field answers the question you are asking. + Read `results` there, not `success` and not the exit code. `check` + lints the path against the whole config rather than your rule alone, + `success` says the run worked rather than that the fixture behaved, + and the exit code follows severity — a `level: error` rule exits 1 on + `fail/` and a `warning` rule exits 0, both correct. `verify` is the + command that answers pass-or-fail; `check` is the one that shows you + the finding. - If `fail/` reports nothing, work down this list before touching the - pattern — the cause is usually further up: + If `fail/` documents show up in `missingFailures`, work down this list + before touching the pattern — the cause is usually further up: - Is the assignment inside a section? - Does the section's glob match the fixture's extension? - Is the rule's `level` at or above `MinAlertLevel`? @@ -332,20 +357,20 @@ A mismatch is not an error. The rule simply never runs. directory all agree? - Only then: does the pattern actually match the text? - If `pass/` reports your rule, the pattern is too broad. Look for a - missing word boundary, an unescaped metacharacter, or a swap key that - also matches the form you meant to allow. + Anything in `unexpectedFindings` means the pattern is too broad. Look + for a missing word boundary, an unescaped metacharacter, or a swap key + that also matches the form you meant to allow. 7. **Report.** Show the three paths you wrote, a one-line summary of what the rule flags, and which files the section scopes it to — the scope is a decision the user should see, not one buried in a config. Note that a whole-project `taskless check` will **not** report your fixtures: `.taskless/` is excluded from the project walk, by design. - The fixtures are exercised by pointing `check` at them, as in step 6. + `verify-rule` is what exercises them. ## Worked rules -Eight rules that work, each paired with the near-miss that fails. Every +Nine rules that work, each paired with the near-miss that fails. Every one was run against the bundled Vale; the "what goes wrong" lines are observed behavior, not warnings in principle. Find the entry closest to your intent and start there. diff --git a/packages/cli/src/rules/owner.ts b/packages/cli/src/rules/owner.ts new file mode 100644 index 0000000..68509ae --- /dev/null +++ b/packages/cli/src/rules/owner.ts @@ -0,0 +1,73 @@ +import { stat } from "node:fs/promises"; +import { join } from "node:path"; + +import { + astGrepRuleFileCandidates, + ENGINE_LAYOUTS, + type EngineName, +} from "./engines"; + +/** + * Which engines have a rule file for `ruleId`. + * + * Ownership is decided by **where the file is**, never by parsing it — the same + * rule `dispatch` follows, so a rule cannot be verified by one engine and run + * by another. `.taskless/vale/rules/.yml` is Vale's; the ast-grep + * candidates (including the pre-`0004` location) are `sg`'s. + * + * Returns every match rather than a single winner. Two engines holding the same + * id is a real state — a project that authored `no-simply` under both — and + * picking one silently would verify a file the user was not asking about. The + * caller reports the ambiguity and names both paths. + * + * A missing directory or a path whose ancestor is a file both read as "not + * here"; anything else is a real IO problem and propagates. + */ +export async function rulefileOwners( + cwd: string, + ruleId: string +): Promise { + const candidates: Array<[EngineName, string[]]> = [ + ["sg", astGrepRuleFileCandidates(cwd, ruleId)], + [ + "vale", + [ + join( + cwd, + ".taskless", + ENGINE_LAYOUTS.vale.rulesDirectory, + `${ruleId}.yml` + ), + ], + ], + ]; + + const owners: EngineName[] = []; + for (const [engine, paths] of candidates) { + for (const path of paths) { + if (await isFile(path)) { + owners.push(engine); + break; + } + } + } + return owners; +} + +/** Whether `path` is an existing regular file. */ +async function isFile(path: string): Promise { + try { + const stats = await stat(path); + return stats.isFile(); + } catch (error) { + const { code } = error as NodeJS.ErrnoException; + if (code === "ENOENT" || code === "ENOTDIR") return false; + throw error; + } +} + +/** Where a rule of each engine lives, for an error message that can be acted on. */ +export function ruleFileLocation(engine: EngineName, ruleId: string): string { + const layout = ENGINE_LAYOUTS[engine]; + return `.taskless/${layout.rulesDirectory}/${ruleId}.yml`; +} diff --git a/packages/cli/src/schemas/rules-verify.ts b/packages/cli/src/schemas/rules-verify.ts index 39187aa..fa2f9ea 100644 --- a/packages/cli/src/schemas/rules-verify.ts +++ b/packages/cli/src/schemas/rules-verify.ts @@ -42,6 +42,9 @@ const testLayerResultSchema = layerResultSchema.extend({ }); export const verifyOutputSchema = z.object({ + engine: z + .literal("sg") + .describe("The engine that owns this rule, decided by where its file lives"), success: z.boolean().describe("True if all layers passed"), ruleId: z.string(), schema: layerResultSchema.describe("Layer 1: Zod schema validation"), @@ -51,6 +54,39 @@ export const verifyOutputSchema = z.object({ tests: testLayerResultSchema.describe("Layer 3: sg test execution"), }); +/** + * Vale verification output. + * + * Deliberately not squeezed into the ast-grep shape. `sg` verification is three + * layers over a rule file and its test cases; Vale verification is one question + * asked of two fixture buckets — did every `fail/` document fire, did every + * `pass/` document stay quiet. Mapping the second onto `schema`/`requirements`/ + * `tests` would invent two empty layers and lose the fixture coverage, which is + * the part that catches a rule that was never really verified. + * + * `engine` is the discriminant. A consumer branches on it before reading + * anything else, and adding an engine cannot silently change the meaning of a + * field another engine already emits. + */ +export const valeVerifyOutputSchema = z.object({ + engine: z.literal("vale"), + success: z + .boolean() + .describe("True only when both buckets are populated and both behaved"), + ruleId: z.string(), + fixtures: z + .enum(["both", "pass-only", "fail-only", "none"]) + .describe( + "Which fixture buckets held documents. Only 'both' can succeed: one bucket alone is half a claim" + ), + missingFailures: z + .array(z.string()) + .describe("fail/ fixtures the rule should have flagged and did not"), + unexpectedFindings: z + .array(z.string()) + .describe("pass/ fixtures the rule flagged and should not have"), +}); + export const verifyErrorSchema = z.object({ success: z.literal(false), error: z.string().describe("Error message"), diff --git a/packages/cli/src/types/errors.ts b/packages/cli/src/types/errors.ts index 9711e6d..efc25b8 100644 --- a/packages/cli/src/types/errors.ts +++ b/packages/cli/src/types/errors.ts @@ -15,6 +15,11 @@ export type CLIErrorCode = | "INVALID_INPUT" | "NETWORK_ERROR" | "SCAN_FAILED" + // An engine binary is not present on this host — an ordinary state on an + // unsupported architecture, and distinct from the engine running and + // failing. Kept apart from SCAN_FAILED because the two ask the caller for + // different things: install something, versus fix something. + | "ENGINE_UNAVAILABLE" | "RECONCILE_FAILED" | "SCAFFOLD_VERSION_MISMATCH" | "SCAFFOLD_CONFLICT" diff --git a/packages/cli/test/rule-verify-dispatch.test.ts b/packages/cli/test/rule-verify-dispatch.test.ts new file mode 100644 index 0000000..926abcc --- /dev/null +++ b/packages/cli/test/rule-verify-dispatch.test.ts @@ -0,0 +1,277 @@ +import { execFile } from "node:child_process"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { promisify } from "node:util"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; + +import { findValeBinary } from "../src/rules/vale/binary"; + +const execFileAsync = promisify(execFile); +const binPath = resolve(import.meta.dirname, "../dist/index.js"); + +/** + * Vale ships as a platform binary, so a host without one (an unsupported arch, + * a blocked install) cannot run these. Skipping is right where the alternative + * is a suite that fails for a reason unrelated to the code under test. + */ +const withVale = findValeBinary().path === undefined ? describe.skip : describe; + +async function runCli( + args: string[] +): Promise<{ stdout: string; stderr: string; exitCode: number }> { + try { + const { stdout, stderr } = await execFileAsync("node", [binPath, ...args]); + return { stdout, stderr, exitCode: 0 }; + } catch (error) { + const execError = error as { stdout: string; stderr: string; code: number }; + return { + stdout: execError.stdout ?? "", + stderr: execError.stderr ?? "", + exitCode: execError.code, + }; + } +} + +const VALE_RULE = `extends: existence +message: "Avoid %s" +level: warning +tokens: + - simply +`; + +const VALE_CONFIG = `StylesPath = . +MinAlertLevel = suggestion + +[*.md] +BasedOnStyles = +rules.no-simply = YES +`; + +/** Scaffold a project with a Vale rule and both fixture buckets populated. */ +async function scaffoldValeRule(cwd: string): Promise { + const vale = join(cwd, ".taskless", "vale"); + await mkdir(join(vale, "rules"), { recursive: true }); + await mkdir(join(vale, "rule-tests", "no-simply", "pass"), { + recursive: true, + }); + await mkdir(join(vale, "rule-tests", "no-simply", "fail"), { + recursive: true, + }); + await writeFile(join(vale, "rules", "no-simply.yml"), VALE_RULE); + await writeFile(join(vale, ".vale.ini"), VALE_CONFIG); + await writeFile( + join(vale, "rule-tests", "no-simply", "fail", "bad.md"), + "You simply do it.\n" + ); + await writeFile( + join(vale, "rule-tests", "no-simply", "pass", "ok.md"), + "You do it.\n" + ); +} + +interface ValeVerifyJson { + engine: string; + success: boolean; + ruleId: string; + fixtures: string; + missingFailures: string[]; + unexpectedFindings: string[]; +} + +describe("rule verify dispatches by owning engine", () => { + let cwd: string; + + beforeEach(async () => { + cwd = await mkdtemp(join(tmpdir(), "taskless-verify-dispatch-")); + }); + + afterEach(async () => { + await rm(cwd, { recursive: true, force: true }); + }); + + // The id lives under both engines. Verifying one silently would report on a + // file the caller may not have meant, so this is an error naming both. + it("refuses an id that exists under two engines", async () => { + await scaffoldValeRule(cwd); + await mkdir(join(cwd, ".taskless", "sg", "rules"), { recursive: true }); + await writeFile( + join(cwd, ".taskless", "sg", "rules", "no-simply.yml"), + "id: no-simply\nlanguage: TypeScript\nseverity: error\nmessage: x\nrule:\n pattern: eval($A)\n" + ); + + const result = await runCli(["rule", "verify", "no-simply", "-d", cwd]); + expect(result.exitCode).not.toBe(0); + expect(result.stderr).toContain("more than one engine"); + expect(result.stderr).toContain(".taskless/sg/rules/no-simply.yml"); + expect(result.stderr).toContain(".taskless/vale/rules/no-simply.yml"); + }); + + it("tags the ast-grep result with its engine", async () => { + await mkdir(join(cwd, ".taskless", "sg", "rules"), { recursive: true }); + await writeFile( + join(cwd, ".taskless", "sg", "rules", "no-eval.yml"), + "id: no-eval\nlanguage: TypeScript\nseverity: error\nmessage: no eval\nrule:\n pattern: eval($$$A)\n" + ); + + const result = await runCli([ + "rule", + "verify", + "no-eval", + "-d", + cwd, + "--json", + ]); + const parsed = JSON.parse(result.stdout) as { engine: string }; + expect(parsed.engine).toBe("sg"); + }); +}); + +withVale("rule verify enforces the Vale fixture layout", () => { + let cwd: string; + + beforeEach(async () => { + cwd = await mkdtemp(join(tmpdir(), "taskless-vale-verify-")); + await scaffoldValeRule(cwd); + }); + + afterEach(async () => { + await rm(cwd, { recursive: true, force: true }); + }); + + it("passes a rule that fires on fail/ and stays quiet on pass/", async () => { + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).toBe(0); + const parsed = JSON.parse(result.stdout) as ValeVerifyJson; + expect(parsed.engine).toBe("vale"); + expect(parsed.success).toBe(true); + expect(parsed.fixtures).toBe("both"); + }); + + // A one-sided rule is the failure this enforcement exists for. With only + // `fail/` fixtures the rule was never shown not to over-fire, and reporting + // that as a pass is how an unverified rule ships looking verified. + it("fails a rule with only fail/ fixtures", async () => { + await rm(join(cwd, ".taskless", "vale", "rule-tests", "no-simply", "pass"), { + recursive: true, + force: true, + }); + + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).not.toBe(0); + const parsed = JSON.parse(result.stdout) as ValeVerifyJson; + expect(parsed.success).toBe(false); + expect(parsed.fixtures).toBe("fail-only"); + }); + + it("fails a rule with only pass/ fixtures", async () => { + await rm(join(cwd, ".taskless", "vale", "rule-tests", "no-simply", "fail"), { + recursive: true, + force: true, + }); + + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).not.toBe(0); + const parsed = JSON.parse(result.stdout) as ValeVerifyJson; + expect(parsed.success).toBe(false); + expect(parsed.fixtures).toBe("pass-only"); + }); + + // Vale lints the rule's whole tree, so a nested document is linted but never + // checked against a bucket. Rejecting names the path at the moment someone + // creates it, rather than letting half a rule's fixtures go unverified. + it("rejects a nested fixture directory by name", async () => { + const nested = join( + cwd, + ".taskless", + "vale", + "rule-tests", + "no-simply", + "pass", + "nested" + ); + await mkdir(nested, { recursive: true }); + await writeFile(join(nested, "x.md"), "fine\n"); + + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).not.toBe(0); + const parsed = JSON.parse(result.stdout) as { + ok: boolean; + code: string; + message: string; + }; + expect(parsed.ok).toBe(false); + expect(parsed.message).toContain("flat"); + expect(parsed.message).toContain("nested"); + }); + + it("reports which fail/ fixture did not fire", async () => { + await writeFile( + join(cwd, ".taskless", "vale", "rule-tests", "no-simply", "fail", "b.md"), + "This text does not contain the token.\n" + ); + + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).not.toBe(0); + const parsed = JSON.parse(result.stdout) as ValeVerifyJson; + expect(parsed.success).toBe(false); + expect(parsed.missingFailures).toHaveLength(1); + expect(parsed.missingFailures[0]).toContain("b.md"); + }); + + it("reports which pass/ fixture wrongly fired", async () => { + await writeFile( + join(cwd, ".taskless", "vale", "rule-tests", "no-simply", "pass", "b.md"), + "You simply cannot.\n" + ); + + const result = await runCli([ + "rule", + "verify", + "no-simply", + "-d", + cwd, + "--json", + ]); + expect(result.exitCode).not.toBe(0); + const parsed = JSON.parse(result.stdout) as ValeVerifyJson; + expect(parsed.success).toBe(false); + expect(parsed.unexpectedFindings).toHaveLength(1); + expect(parsed.unexpectedFindings[0]).toContain("b.md"); + }); +}); From 1994f9aea484524e7ee3dd2d5bd46181f1fc96f4 Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Fri, 14 Aug 2026 19:25:36 -0700 Subject: [PATCH 18/19] feat(cli)!: rename `taskless help` to `taskless agent`, single-token topics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The surface an agent reads was shaped for a human. `taskless help` names the command after a human's reason for typing it, and addressing was longform, so an agent had to know both the words and their order. Sweeps the shipped surface: 17 files, every recipe, both READMEs, the skill, and the prompts doc comment. Nested topics flatten to their filenames (`rule improve` → `improve-rule`). `static`, `existing`, and `engine-selection` are removed rather than renamed, because `route` now applies the criterion itself and names a concrete destination. Left alone deliberately: CHANGELOG.md, openspec/changes/archive/**, and this change's own proposal and design, which quote the old name to describe the rename. Three cross-reference tests, and each caught something. Five recipes still announced themselves by their old multi-token name, so `taskless agent delete-rule` printed "# Topic: rule delete". The See Also check found a dangling entry left by the sweep collapsing two topics onto `route`. The `taskless help` check is the cheap total one the design asked for. Verified by rehearsing `route` against fresh agents with no repository access, one per destination: all five route correctly, and the two judgment branches (present-both when logged in and locally expressible, split on login state for runtime) behave as written. Also adds the W101 notice test and its no-matcher counterpart, which is what keeps the section-less scaffold from reintroducing a silent disable. BREAKING CHANGE: `taskless help ` is now `taskless agent `, topics are single-token, and `@taskless/cli/prompts` no longer exports `engine-selection`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .changeset/vale-rule-engine.md | 45 +++++++--- README.md | 2 +- .../agent-command-and-vale-authoring/tasks.md | 29 ++++--- packages/cli/README.md | 8 +- packages/cli/src/help/auth.txt | 4 +- packages/cli/src/help/check.txt | 6 +- packages/cli/src/help/ci.txt | 10 +-- packages/cli/src/help/delete-rule.txt | 6 +- packages/cli/src/help/detect.txt | 11 ++- .../cli/src/help/improve-rule.anonymous.txt | 10 +-- packages/cli/src/help/improve-rule.txt | 18 ++-- packages/cli/src/help/info.txt | 4 +- packages/cli/src/help/init.txt | 4 +- packages/cli/src/help/onboard.txt | 10 +-- packages/cli/src/help/rule-meta.txt | 4 +- packages/cli/src/help/rule.txt | 28 +++--- packages/cli/src/help/update.txt | 4 +- packages/cli/src/help/verify-rule.txt | 6 +- packages/cli/src/prompts/index.ts | 2 +- packages/cli/test/help-extensions.test.ts | 2 +- packages/cli/test/mixed-engine-check.test.ts | 86 +++++++++++++++++++ .../cli/test/recipe-cross-references.test.ts | 85 ++++++++++++++++++ skills/taskless/SKILL.md | 2 +- 23 files changed, 291 insertions(+), 95 deletions(-) create mode 100644 packages/cli/test/recipe-cross-references.test.ts diff --git a/.changeset/vale-rule-engine.md b/.changeset/vale-rule-engine.md index 11f1e29..9079c3c 100644 --- a/.changeset/vale-rule-engine.md +++ b/.changeset/vale-rule-engine.md @@ -2,16 +2,39 @@ "@taskless/cli": minor --- -Add Vale as a second static-tier rule engine. +Add Vale as a second static-tier rule engine, and rename the agent-facing command. -`check` now dispatches by engine directory and runs ast-grep, Vale, and runtime -rules concurrently, merging their findings into one result set. Vale rules live -in `.taskless/vale/` and execute against the committed `.vale.ini`; an -unavailable Vale reports itself and the other engines still return, while a Vale -that times out or rejects its config fails the check rather than passing as a -clean run. Vale rules are verified from `rule-tests//pass|fail` fixtures -against a generated per-rule config. +`check` now dispatches by engine and runs ast-grep, Vale, and runtime rules +concurrently, merging their findings into one result set. An unavailable Vale +reports itself and the other engines still return. A Vale that times out or +rejects its config fails the check rather than passing as a clean run. -Adds the `engine-selection` knowledge topic — which engine enforces a given -rule, and why — available from `taskless help engine-selection` and exported -through `@taskless/cli/prompts`. +**BREAKING: `taskless help ` is now `taskless agent `.** The +command is named for who reads it. Agents fetching a procedure are not asking +for help, and the old name is gone rather than aliased. + +**BREAKING: topics are addressed by a single token.** `taskless help rule +create` becomes `taskless agent create-sg-rule`; multiple positionals are no +longer joined into a topic key. A topic name is now a literal string an agent +copies rather than a phrase it can reorder. The renames: + +| Was | Now | +| -------------------- | --------------------------------------- | +| `rule create` | `create-sg-rule` / `create-remote-rule` | +| `rule improve` | `improve-rule` | +| `rule delete` | `delete-rule` | +| `rule verify` | `verify-rule` | +| `rule meta` | `rule-meta` | +| `static`, `existing` | `route` | +| `engine-selection` | `route` | + +`route` now applies the engine reasoning itself and names a concrete +`create-*-rule` topic, so `engine-selection`, `static`, and `existing` are +removed rather than renamed. Their criterion is stated once, in `route`. + +**BREAKING for `@taskless/cli/prompts` consumers.** `engine-selection` is no +longer exported. `TOPICS` is now `create-sg-rule`, `create-vale-rule`, and +`create-runtime-rule`, so a consumer that decides an engine can reach the +procedure for each destination. Because the export is a string union, a +consumer passing the removed name dynamically breaks on upgrade rather than at +build time. diff --git a/README.md b/README.md index 26c6748..56c5dc1 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Starting in v0.7, Taskless ships a **single consolidated skill** (`taskless`) pl | | | delete, check, auth, CI). Fetches the canonical recipe | | | | for the user's intent and follows it. | -Available `taskless help` topics: `rule create`, `rule improve`, `rule delete`, `check`, `auth`, `ci`, `info`, `init`, `update`. Append `--anonymous` for the local-only flow on rule create/improve. +Available `taskless agent` topics: `route`, `create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, `create-remote-rule`, `improve-rule`, `delete-rule`, `check`, `auth`, `ci`, `info`, `init`, `update`. Run `taskless agent` with no topic for the index. Append `--anonymous` for the local-only flow on improve. ## CLI diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/agent-command-and-vale-authoring/tasks.md index 28d6b26..2aabb87 100644 --- a/openspec/changes/agent-command-and-vale-authoring/tasks.md +++ b/openspec/changes/agent-command-and-vale-authoring/tasks.md @@ -38,32 +38,33 @@ The recipes are the deliverable, and a recipe that reads well to its author whil - [x] 2b.5 Iterate across three intents that exercise different extension points — one `existence`, one `substitution` (prefer X over Y), one `capitalization` (headings, product names). Everything in this repo today is `existence`, so a recipe drafted from our own examples teaches token blocklists and nothing else. Vale has eleven extension points and most real prose rules are not blocklists - [x] 2b.6 Every failure is a defect in the prose, not in the agent. Fix the recipe and re-run with a fresh agent. Converged when an agent, given an intent the recipe never names, produces a rule that fires on its `fail` fixture and stays quiet on its `pass` fixture, first try, uncorrected - [x] 2b.7 Keep the iteration log — what failed, what changed, what finally held. It is the evidence the prose works, and the only part of this a reviewer can check without rerunning the loop -- [ ] 2b.8 Run the same harness over `create-sg-rule` as a control. It documents a flow that already works, so a failure there means the harness is wrong rather than the recipe +- [x] 2b.8 Run the same harness over `create-sg-rule` as a control. It documents a flow that already works, so a failure there means the harness is wrong rather than the recipe ## 3. Sweep the cross-references -- [ ] 3.1 Replace every `taskless help ` occurrence with `taskless agent ` across `src/help/*.txt`, `src/**/*.ts`, `skills/taskless/SKILL.md`, `README.md`, and `packages/cli/README.md` (~306 occurrences, 77 files). Leave `CHANGELOG.md` alone — it is a historical record -- [ ] 3.2 Update every reference to a renamed topic (`static`, `existing`, `rule create`, …) to its new single-token name -- [ ] 3.3 Add a test asserting no shipped recipe contains the string `taskless help`, and that every topic named in a recipe's See Also resolves to an embedded file. A stale cross-reference is otherwise invisible until an agent runs it +- [x] 3.1 Replace every `taskless help ` occurrence with `taskless agent ` across `src/help/*.txt`, `src/**/*.ts`, `skills/taskless/SKILL.md`, `README.md`, and `packages/cli/README.md` (~306 occurrences, 77 files). Leave `CHANGELOG.md` alone — it is a historical record +- [x] 3.2 Update every reference to a renamed topic (`static`, `existing`, `rule create`, …) to its new single-token name +- [x] 3.3 Add a test asserting no shipped recipe contains the string `taskless help`, and that every topic named in a recipe's See Also resolves to an embedded file. A stale cross-reference is otherwise invisible until an agent runs it ## 4. Update the export surface -- [ ] 4.1 `TOPICS` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]` and no longer exports `engine-selection`; move the renamed authoring topics through `INTERNAL_TOPICS` as their membership requires, keeping the two lists disjoint and jointly exhaustive over the recipe files -- [ ] 4.2 Update `prompts.test.ts` — the membership test compares against the files on disk, so it fails until the rename is complete in both places -- [ ] 4.3 Write the changeset as **MINOR** — pre-1.0, backwards-incompatible is MINOR — naming the removed topic names explicitly and stating that `@taskless/cli/prompts` consumers break on upgrade rather than at build time +- [x] 4.1 `TOPICS` becomes `["create-sg-rule", "create-vale-rule", "create-runtime-rule"]` and no longer exports `engine-selection`; move the renamed authoring topics through `INTERNAL_TOPICS` as their membership requires, keeping the two lists disjoint and jointly exhaustive over the recipe files +- [x] 4.2 Update `prompts.test.ts` — the membership test compares against the files on disk, so it fails until the rename is complete in both places +- [x] 4.3 Write the changeset as **MINOR** — pre-1.0, backwards-incompatible is MINOR — naming the removed topic names explicitly and stating that `@taskless/cli/prompts` consumers break on upgrade rather than at build time ## 5. Scaffold and diagnostics (ships together) - [x] 5.1 `VALE_CONFIG_CONTENT` in `0004-vale-engine.ts` drops its `[*]` section, leaving `StylesPath` and `MinAlertLevel` - [x] 5.2 `runVale` captures stderr on a zero-exit run and returns it as a notice on the `ok` outcome; `runValeEngine` forwards it to `DispatchResult.notices`. A notice must not touch the exit code - **Pulled forward out of order, deliberately.** 2b tests `create-vale-rule` against a scaffolded project, and the recipe's central claim is that the scaffold ships section-less. Running the harness against a scaffold that still wrote `[*]` would have tested a recipe nobody will receive. Measured end to end afterwards: section-less scaffold + a top-level `rules. = YES` now prints `Notice: Vale reported while running: W101 'rules.no-simply' isn't a core option; Vale is ignoring it.` and exits 0 -- [ ] 5.3 Test that a rule enabled outside a section produces a notice containing Vale's `W101` text and exits zero — this is the pairing that keeps 5.1 from reintroducing a silent disable -- [ ] 5.4 Extend the mixed-engine integration test: a scaffolded project with a rule file and no section reports nothing and does not fail; adding a section makes the same rule fire +- [x] 5.3 Test that a rule enabled outside a section produces a notice containing Vale's `W101` text and exits zero — this is the pairing that keeps 5.1 from reintroducing a silent disable +- [x] 5.4 Extend the mixed-engine integration test: a scaffolded project with a rule file and no section reports nothing and does not fail; adding a section makes the same rule fire + - **Superseded immediately above this change in the stack.** `self-contained-rules` gives every Vale rule its own config, so "a rule file and no section" stops being a quiet no-op and becomes a `verify` error: nothing scopes it, so it can never run, and saying so is better than reporting nothing. This test is correct for the layout this change ships and is replaced there rather than carried forward. The W101 pairing in 5.3 does survive, because an assignment can still be written above its own matcher. ## 6. Verify -- [ ] 6.1 `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli build`, `pnpm --filter @taskless/cli test` -- [ ] 6.2 **Rehearse `route` against a fresh agent.** Hand it the text with no prior context and a request to author a rule, then check which destination it names and why. Unlike the authoring recipes (2b), `route` produces a decision rather than artifacts, so the plan it describes is the only thing there is to check. Cover one case per destination, including a logged-out runtime request -- [ ] 6.2a Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero -- [ ] 6.3 `pnpm openspec validate --all --strict` (note: `cli-rules` and `cli-update-engine` fail on `main` already and are unrelated) -- [ ] 6.4 Archive the change +- [x] 6.1 `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli build`, `pnpm --filter @taskless/cli test` +- [x] 6.2 **Rehearse `route` against a fresh agent.** Hand it the text with no prior context and a request to author a rule, then check which destination it names and why. Unlike the authoring recipes (2b), `route` produces a decision rather than artifacts, so the plan it describes is the only thing there is to check. Cover one case per destination, including a logged-out runtime request +- [x] 6.2a Run `taskless agent` with no argument, with each renamed topic, and with a removed name, confirming the index lists the new vocabulary and a removed name exits non-zero +- [x] 6.3 `pnpm openspec validate --all --strict` (note: `cli-rules` and `cli-update-engine` fail on `main` already and are unrelated) +- [x] 6.4 Archive the change diff --git a/packages/cli/README.md b/packages/cli/README.md index eb27e36..af0595f 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -36,7 +36,7 @@ current project (`.claude/`, `.opencode/`, `.cursor/`, `.agents/`), asks which tools to enable Taskless for, and walks through the auth tradeoff before writing anything. Running `taskless` with no subcommand in a TTY also launches this wizard. Without a TTY, bare `taskless` prints a short context preamble -followed by the topic index from `taskless help`. +followed by the topic index from `taskless agent`. In v0.7+, there is exactly one skill (`taskless`) and one command (`tskl`) — no opt-in selection needed. @@ -142,10 +142,10 @@ taskless rule delete no-console-log Lists available subcommands. -### `taskless help [topic]` +### `taskless agent [topic]` Returns agent-facing recipes. With no args, prints the topic index. With a -topic (e.g. `taskless help rule create`), prints the full step-by-step recipe +topic (e.g. `taskless agent route`), prints the full step-by-step recipe for that operation, including an embedded JSON Schema for any `--from` input and a table of stable error codes. Append `--anonymous` to fetch the local-only variant where one exists (currently `rule create`/`rule improve`). @@ -159,7 +159,7 @@ relevant recipe on demand. Recognized on every command. Behavior matrix: - `rule create` / `rule improve` — exits with a pointer to - `taskless help --anonymous`. The local-only flow runs in the agent + `taskless agent --anonymous`. The local-only flow runs in the agent per the recipe variant. - `info` — skips the API/auth probe; reports local state only. - `auth login` — rejected (auth commands cannot be anonymous). diff --git a/packages/cli/src/help/auth.txt b/packages/cli/src/help/auth.txt index 7c69bc8..7376aa4 100644 --- a/packages/cli/src/help/auth.txt +++ b/packages/cli/src/help/auth.txt @@ -73,5 +73,5 @@ report. ## See Also -- `taskless help info` — see auth state and skill versions -- `taskless help rule create` — first action that requires auth +- `taskless agent info` — see auth state and skill versions +- `taskless agent route` — first action that requires auth diff --git a/packages/cli/src/help/check.txt b/packages/cli/src/help/check.txt index 9523aa3..fe9e01d 100644 --- a/packages/cli/src/help/check.txt +++ b/packages/cli/src/help/check.txt @@ -40,7 +40,7 @@ only: they never change the exit code. Under `--json` they do NOT appear as warnings; instead an additive optional `skipped: [{ rule, reason }]` array is included alongside the unchanged `{ success, results }`. The authoritative allow-list is the server's; the CI backstop -(`taskless help ci`) is the enforcement point for runtime rules. +(`taskless agent ci`) is the enforcement point for runtime rules. ## Flags - `--json` — machine output (`{ success, results, skipped? }`). @@ -125,5 +125,5 @@ When `--json` is set, failures emit `{ ok: false, code, message }`: ## See Also -- `taskless help rule create` — add a rule if none exist -- `taskless help ci` — wire `check` into a CI pipeline +- `taskless agent route` — add a rule if none exist +- `taskless agent ci` — wire `check` into a CI pipeline diff --git a/packages/cli/src/help/ci.txt b/packages/cli/src/help/ci.txt index 64f5d1c..3164cd9 100644 --- a/packages/cli/src/help/ci.txt +++ b/packages/cli/src/help/ci.txt @@ -12,7 +12,7 @@ you recognize one not on the list, apply the same patterns. ## Preconditions - `.taskless/` directory exists and contains at least one rule. (If no rules exist, instruct the user to fetch - `taskless help rule create` first — wiring CI with zero rules + `taskless agent route` first — wiring CI with zero rules produces an always-green check that gives false confidence.) - A local `taskless check` succeeds (or fails with real findings the user is OK with seeing in CI's first run). @@ -66,7 +66,7 @@ main. Run `npx @taskless/cli check`: - Clean pass → proceed. -- "No rules configured" → stop. Fetch `taskless help rule create`. +- "No rules configured" → stop. Fetch `taskless agent route`. - Findings → tell the user CI will fail; ask whether to fix, suppress, or proceed knowing the first CI run will be red. @@ -207,7 +207,7 @@ Show: ## Errors -- **No rules** → fetch `taskless help rule create`. Don't write CI +- **No rules** → fetch `taskless agent route`. Don't write CI config. - **Unrecognized CI** → produce a generic `.taskless/ci/check.sh` script implementing the six universal steps. Be upfront it's a @@ -217,5 +217,5 @@ Show: ## See Also -- `taskless help check` — the command being wired into CI -- `taskless help rule create` — required if no rules exist yet +- `taskless agent check` — the command being wired into CI +- `taskless agent route` — required if no rules exist yet diff --git a/packages/cli/src/help/delete-rule.txt b/packages/cli/src/help/delete-rule.txt index 1bdec30..108ad16 100644 --- a/packages/cli/src/help/delete-rule.txt +++ b/packages/cli/src/help/delete-rule.txt @@ -1,4 +1,4 @@ -# Topic: rule delete (CLI v%(CLI_VERSION)s / topic v1) +# Topic: delete-rule (CLI v%(CLI_VERSION)s / topic v2) ## Goal Remove a rule and its associated test files from `.taskless/`. Does @@ -44,5 +44,5 @@ emitted from this command. ## See Also -- `taskless help rule create` — make a new rule -- `taskless help check` — run remaining rules to confirm nothing broke +- `taskless agent route` — make a new rule +- `taskless agent check` — run remaining rules to confirm nothing broke diff --git a/packages/cli/src/help/detect.txt b/packages/cli/src/help/detect.txt index a803605..fdc399d 100644 --- a/packages/cli/src/help/detect.txt +++ b/packages/cli/src/help/detect.txt @@ -45,9 +45,9 @@ routing flow reads `detect` to decide where a new rule should live. 3. **Use the signals to route.** Feed the output into rule authoring: - A detected linter the repo already uses → author the rule there - (`taskless help existing`). - - No suitable linter, local-only → `taskless help static`. - - See `taskless help route` for the full decision. + (`taskless agent route`). + - No suitable linter, local-only → `taskless agent create-sg-rule`. + - See `taskless agent route` for the full decision. ## Errors @@ -59,6 +59,5 @@ When `--json` is set, failures emit `{ ok: false, code, message }`: ## See Also -- `taskless help route` — decide where to author a rule from these signals -- `taskless help existing` — author a rule in a linter the repo already uses -- `taskless help check` — run rules against the codebase +- `taskless agent route` — decide where to author a rule from these signals +- `taskless agent check` — run rules against the codebase diff --git a/packages/cli/src/help/improve-rule.anonymous.txt b/packages/cli/src/help/improve-rule.anonymous.txt index d2160cb..e719f4c 100644 --- a/packages/cli/src/help/improve-rule.anonymous.txt +++ b/packages/cli/src/help/improve-rule.anonymous.txt @@ -1,4 +1,4 @@ -# Topic: rule improve (anonymous) (CLI v%(CLI_VERSION)s / topic v1) +# Topic: improve-rule (anonymous) (CLI v%(CLI_VERSION)s / topic v2) ## Goal Iterate on an existing ast-grep rule **locally** without contacting @@ -59,7 +59,7 @@ validate with `rule verify` in a feedback loop. - The updated rule file path - The updated test file path - A diff-style summary of what changed - Suggest fetching `taskless help check` to validate against the + Suggest fetching `taskless agent check` to validate against the broader codebase. ## Important Notes @@ -83,6 +83,6 @@ The verify primitive returns structured errors per layer: ## See Also -- `taskless help rule improve` — API-backed flow (auth required) -- `taskless help rule create --anonymous` — make a new rule locally -- `taskless help check` — validate the updated rule +- `taskless agent improve-rule` — API-backed flow (auth required) +- `taskless agent create-sg-rule` — make a new rule locally +- `taskless agent check` — validate the updated rule diff --git a/packages/cli/src/help/improve-rule.txt b/packages/cli/src/help/improve-rule.txt index 5c12f0a..d727eea 100644 --- a/packages/cli/src/help/improve-rule.txt +++ b/packages/cli/src/help/improve-rule.txt @@ -1,4 +1,4 @@ -# Topic: rule improve (CLI v%(CLI_VERSION)s / topic v1) +# Topic: improve-rule (CLI v%(CLI_VERSION)s / topic v2) ## Goal Iterate on an existing Taskless rule. The CLI submits the user's @@ -8,7 +8,7 @@ is to gather the right ruleId + guidance + supporting references and to report the result. If the user wants the local-only flow (no API call), fetch -`taskless help rule improve --anonymous` instead. +`taskless agent improve-rule --anonymous` instead. ## Preconditions - User is logged in. @@ -22,7 +22,7 @@ If the user wants the local-only flow (no API call), fetch ## Steps 1. **Confirm auth.** Run `npx @taskless/cli info --json` and check - `loggedIn`. If false, fetch `taskless help auth`. + `loggedIn`. If false, fetch `taskless agent auth`. 2. **Identify the rule to improve.** If the user named one, use it. Otherwise, list rules in `.taskless/sg/rules/` and ask which one. @@ -34,7 +34,7 @@ If the user wants the local-only flow (no API call), fetch ``` This returns the `ticketId` needed for the iterate request. If the metadata is missing, the rule cannot be iterated via API — fetch - `taskless help rule improve --anonymous` instead. + `taskless agent improve-rule --anonymous` instead. 4. **Gather improvement guidance.** Ask the user what should change: - Are there false positives we need to exclude? @@ -69,7 +69,7 @@ If the user wants the local-only flow (no API call), fetch 10. **Report results.** The CLI overwrites the rule file (and its test file) with the updated version. Show the file paths and a - summary of what changed. Suggest fetching `taskless help check` + summary of what changed. Suggest fetching `taskless agent check` to validate. ## Input schema @@ -89,7 +89,7 @@ When `--json` is set, failures emit `{ ok: false, code, message }`: | code | meaning | fix | |--------------------------|----------------------------------------|----------------------------------------------| -| `AUTH_REQUIRED` | not logged in | fetch `taskless help auth` | +| `AUTH_REQUIRED` | not logged in | fetch `taskless agent auth` | | `NO_GITHUB_REMOTE` | no GitHub origin remote | tell the user; we cannot proceed | | `INVALID_INPUT` | `--from` JSON failed validation | re-read input schema, fix, retry | | `RULE_NOT_FOUND` | metadata missing for the given rule | use anonymous variant or recreate via create | @@ -99,6 +99,6 @@ When `--json` is set, failures emit `{ ok: false, code, message }`: ## See Also -- `taskless help rule improve --anonymous` — local-only flow -- `taskless help rule create` — make a new rule from scratch -- `taskless help check` — validate the updated rule +- `taskless agent improve-rule --anonymous` — local-only flow +- `taskless agent route` — make a new rule from scratch +- `taskless agent check` — validate the updated rule diff --git a/packages/cli/src/help/info.txt b/packages/cli/src/help/info.txt index 6870a24..0544de9 100644 --- a/packages/cli/src/help/info.txt +++ b/packages/cli/src/help/info.txt @@ -63,5 +63,5 @@ failing.) ## See Also -- `taskless help auth` — log in / log out / status detail -- `taskless help check` — run rules against the codebase +- `taskless agent auth` — log in / log out / status detail +- `taskless agent check` — run rules against the codebase diff --git a/packages/cli/src/help/init.txt b/packages/cli/src/help/init.txt index a755fb0..04c05a6 100644 --- a/packages/cli/src/help/init.txt +++ b/packages/cli/src/help/init.txt @@ -50,5 +50,5 @@ what was removed. ## See Also -- `taskless help info` — verify what's installed and check staleness -- `taskless help auth` — authenticate after installing +- `taskless agent info` — verify what's installed and check staleness +- `taskless agent auth` — authenticate after installing diff --git a/packages/cli/src/help/onboard.txt b/packages/cli/src/help/onboard.txt index 58105cb..b1ddc28 100644 --- a/packages/cli/src/help/onboard.txt +++ b/packages/cli/src/help/onboard.txt @@ -15,7 +15,7 @@ rules as a bullet list the user can choose to materialize via the is automatically satisfied. - A working repository the agent can read. - No auth required to surface candidates. (Materializing a rule via - `rule create` may require auth — fetch `taskless help auth` if + `rule create` may require auth — fetch `taskless agent auth` if needed at that point.) ## Steps @@ -91,7 +91,7 @@ rules as a bullet list the user can choose to materialize via the 6. **Offer materialization per bullet.** For each bullet, ask whether the user wants to turn it into a real Taskless rule. On - yes, fetch `taskless help rule create` and follow that recipe — + yes, fetch `taskless agent route` and follow that recipe — the user's accepted bullet becomes the rule description input. 7. **Ask before marking onboarding complete.** When the user signals @@ -117,6 +117,6 @@ rules as a bullet list the user can choose to materialize via the ## See Also -- `taskless help rule create` — materialize an accepted bullet into a real rule -- `taskless help check` — validate newly created rules against the codebase -- `taskless help info` — inspect the current `.taskless/taskless.json` state +- `taskless agent route` — materialize an accepted bullet into a real rule +- `taskless agent check` — validate newly created rules against the codebase +- `taskless agent info` — inspect the current `.taskless/taskless.json` state diff --git a/packages/cli/src/help/rule-meta.txt b/packages/cli/src/help/rule-meta.txt index d9361ef..023c0c1 100644 --- a/packages/cli/src/help/rule-meta.txt +++ b/packages/cli/src/help/rule-meta.txt @@ -1,4 +1,4 @@ -# Topic: rule meta (CLI v%(CLI_VERSION)s / topic v1) +# Topic: rule-meta (CLI v%(CLI_VERSION)s / topic v2) ## Goal Read sidecar metadata for an API-generated rule. Used internally by @@ -27,4 +27,4 @@ version, etc. ## See Also -- `taskless help rule improve` — the primary consumer of this command +- `taskless agent improve-rule` — the primary consumer of this command diff --git a/packages/cli/src/help/rule.txt b/packages/cli/src/help/rule.txt index 05499fa..26acc65 100644 --- a/packages/cli/src/help/rule.txt +++ b/packages/cli/src/help/rule.txt @@ -1,22 +1,24 @@ # Topic: rule (CLI v%(CLI_VERSION)s / topic v1) ## Goal -Umbrella for rule operations. Use the specific subcommand for the -action you want. +Umbrella for rule operations. Fetch the topic for the action you want. -## Subcommands +## Topics -| Subcommand | Recipe | -|---------------------|----------------------------------------------| -| Create a rule | `taskless help rule create` | -| Improve a rule | `taskless help rule improve` | -| Delete a rule | `taskless help rule delete` | -| Verify a rule | `taskless help rule verify` (agent-internal) | -| Read rule metadata | `taskless help rule meta` (agent-internal) | +| Action | Recipe | +|---------------------|-----------------------------------------------| +| Create a rule | `taskless agent route` | +| Improve a rule | `taskless agent improve-rule` | +| Delete a rule | `taskless agent delete-rule` | +| Verify a rule | `taskless agent verify-rule` (agent-internal) | +| Read rule metadata | `taskless agent rule-meta` (agent-internal) | -For local-only flows on create/improve, append `--anonymous` to the -help fetch. +`route` is the entry point for authoring: it reads the request and +names the `create-*-rule` topic that fits, so you do not pick an engine +yourself. + +For the local-only flow on improve, append `--anonymous`. ## See Also -- `taskless help check` — run all configured rules +- `taskless agent check` — run all configured rules diff --git a/packages/cli/src/help/update.txt b/packages/cli/src/help/update.txt index 2188663..4bf75b3 100644 --- a/packages/cli/src/help/update.txt +++ b/packages/cli/src/help/update.txt @@ -44,5 +44,5 @@ non-zero with the error message on stderr. ## See Also -- `taskless help init` — interactive variant (wizard with prompts) -- `taskless help info` — verify what's installed and check staleness +- `taskless agent init` — interactive variant (wizard with prompts) +- `taskless agent info` — verify what's installed and check staleness diff --git a/packages/cli/src/help/verify-rule.txt b/packages/cli/src/help/verify-rule.txt index 4cd7aa8..04b143e 100644 --- a/packages/cli/src/help/verify-rule.txt +++ b/packages/cli/src/help/verify-rule.txt @@ -1,4 +1,4 @@ -# Topic: rule verify (CLI v%(CLI_VERSION)s / topic v1) +# Topic: verify-rule (CLI v%(CLI_VERSION)s / topic v2) ## Goal Validate a rule against the ast-grep schema and run its test cases. @@ -54,5 +54,5 @@ attempts and report to the user if still failing. ## See Also -- `taskless help rule create --anonymous` — primary consumer (loop) -- `taskless help rule improve --anonymous` — primary consumer (loop) +- `taskless agent create-sg-rule` — primary consumer (loop) +- `taskless agent improve-rule --anonymous` — primary consumer (loop) diff --git a/packages/cli/src/prompts/index.ts b/packages/cli/src/prompts/index.ts index 7e58548..4a2506c 100644 --- a/packages/cli/src/prompts/index.ts +++ b/packages/cli/src/prompts/index.ts @@ -11,7 +11,7 @@ import { getRecipe, type RecipeOptions } from "./recipes.js"; * Public entry for `@taskless/cli/prompts`. * * Everything here renders through the same embedded recipe text and the same - * render path `taskless help ` serves, so the two surfaces cannot emit + * render path `taskless agent ` serves, so the two surfaces cannot emit * different guidance. Nothing in this graph reaches the CLI runtime: no citty * command tree, no telemetry, no filesystem or network, so a Worker can import * it without dragging the CLI in behind it. diff --git a/packages/cli/test/help-extensions.test.ts b/packages/cli/test/help-extensions.test.ts index a8ca2fa..8ea2d88 100644 --- a/packages/cli/test/help-extensions.test.ts +++ b/packages/cli/test/help-extensions.test.ts @@ -267,7 +267,7 @@ describe("taskless agent --anonymous (variant lookup)", () => { it("returns the canonical recipe when --anonymous is omitted", async () => { const result = await runCli(["agent", "improve-rule", "-d", cwd]); - expect(result.stdout).toContain("# Topic: rule improve"); + expect(result.stdout).toContain("# Topic: improve-rule"); expect(result.stdout).not.toContain("(anonymous)"); }); }); diff --git a/packages/cli/test/mixed-engine-check.test.ts b/packages/cli/test/mixed-engine-check.test.ts index c60995f..14a51d2 100644 --- a/packages/cli/test/mixed-engine-check.test.ts +++ b/packages/cli/test/mixed-engine-check.test.ts @@ -271,6 +271,92 @@ describe("check over a project with both engines", () => { await rm(scaffold, { recursive: true, force: true }); } }); + + // The pairing that keeps the section-less scaffold safe. Dropping `[*]` + // from the scaffold means scope is the author's decision, which is right, + // and it also means the first rule someone writes can land its assignment + // above every matcher. Vale does not error on that: it warns on stderr, + // exits zero, and returns a well-formed empty result, which is + // indistinguishable from a clean run. Surfacing the warning is the only + // thing standing between that mistake and a silently disabled rule. + it("surfaces Vale's W101 when an assignment sits outside every matcher", async () => { + const scaffold = await mkdtemp(join(tmpdir(), "taskless-w101-")); + try { + const init = await runCli(["init", "--no-interactive", "-d", scaffold]); + expect(init.exitCode).toBe(0); + + const valeDirectory = join(scaffold, ".taskless", "vale"); + await writeFile( + join(valeDirectory, "rules", "no-simply.yml"), + "extends: existence\nmessage: \"Avoid 'simply'\"\nlevel: warning\ntokens:\n - simply\n" + ); + // The mistake: enabled, but above the first `[…]` line, so it belongs + // to no matcher. + const config = await readFile(join(valeDirectory, ".vale.ini"), "utf8"); + await writeFile( + join(valeDirectory, ".vale.ini"), + `${config}\nrules.no-simply = YES\n` + ); + await writeFile(join(scaffold, "doc.md"), "Just simply do it.\n"); + + const { stdout, stderr, exitCode } = await runCli([ + "check", + "-d", + scaffold, + ]); + + // Advisory, so the run still succeeds: nothing is broken, something is + // misplaced. Failing here would make an ignorable warning block a check. + expect(exitCode).toBe(0); + expect(`${stdout}${stderr}`).toContain("W101"); + expect(`${stdout}${stderr}`).toContain("no-simply"); + } finally { + await rm(scaffold, { recursive: true, force: true }); + } + }); + + // The other half: a rule file with no matcher enabling it at all. This is + // the state a scaffold plus a half-finished rule leaves behind, and `check` + // must neither report it nor fail on it — reporting would be a lie and + // failing would break a project mid-authoring. + it("reports nothing and does not fail when no matcher enables the rule", async () => { + const scaffold = await mkdtemp(join(tmpdir(), "taskless-unscoped-")); + try { + const init = await runCli(["init", "--no-interactive", "-d", scaffold]); + expect(init.exitCode).toBe(0); + await writeFile( + join(scaffold, ".taskless", "vale", "rules", "no-simply.yml"), + "extends: existence\nmessage: \"Avoid 'simply'\"\nlevel: warning\ntokens:\n - simply\n" + ); + await writeFile(join(scaffold, "doc.md"), "Just simply do it.\n"); + + const quiet = await runCli(["check", "-d", scaffold, "--json"]); + expect(quiet.exitCode).toBe(0); + expect( + (JSON.parse(quiet.stdout.trim()) as CheckOutput).results.filter( + (finding) => finding.source === "vale" + ) + ).toEqual([]); + + // Adding the matcher is the whole difference. Same rule, same + // document, now scoped. + const configPath = join(scaffold, ".taskless", "vale", ".vale.ini"); + await writeFile( + configPath, + `${await readFile(configPath, "utf8")}\n[*.md]\nBasedOnStyles =\nrules.no-simply = YES\n` + ); + + const loud = await runCli(["check", "-d", scaffold, "--json"]); + expect( + (JSON.parse(loud.stdout.trim()) as CheckOutput).results.some( + (finding) => + finding.source === "vale" && finding.ruleId === "no-simply" + ) + ).toBe(true); + } finally { + await rm(scaffold, { recursive: true, force: true }); + } + }); }); describe("whatever the host provides", () => { diff --git a/packages/cli/test/recipe-cross-references.test.ts b/packages/cli/test/recipe-cross-references.test.ts new file mode 100644 index 0000000..8311da9 --- /dev/null +++ b/packages/cli/test/recipe-cross-references.test.ts @@ -0,0 +1,85 @@ +import { readdir, readFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { describe, expect, it } from "vitest"; + +/** + * Recipes cite each other by topic name, in prose. Nothing resolves those + * citations at build time — they are strings an agent reads and then types — + * so a renamed topic leaves a reference that stays valid-looking and fails + * only when an agent runs it and gets a non-zero exit and no recipe. + * + * That is the failure mode the `taskless help` → `taskless agent` rename + * created 90 opportunities for. These read the recipe sources, which are the + * authority for what ships, rather than a built bundle: the bundler resolves + * imports, and a prose cross-reference is not one, so there is no structured + * answer to ask it for. + */ +const helpDirectory = resolve(import.meta.dirname, "../src/help"); + +async function recipeFiles(): Promise { + const entries = await readdir(helpDirectory); + return entries.filter((entry) => entry.endsWith(".txt")).toSorted(); +} + +/** Topic names that resolve, i.e. `.txt` exists in the help directory. */ +async function embeddedTopics(): Promise> { + const files = await recipeFiles(); + return new Set( + files + .map((file) => file.replace(/\.txt$/, "")) + .map((name) => name.replace(/\.anonymous$/, "")) + ); +} + +describe("shipped recipes name only commands that exist", () => { + it("contains no reference to the former `taskless help` command", async () => { + const offenders: string[] = []; + for (const file of await recipeFiles()) { + const source = await readFile(join(helpDirectory, file), "utf8"); + for (const [index, line] of source.split("\n").entries()) { + if (line.includes("taskless help")) { + offenders.push(`${file}:${String(index + 1)}: ${line.trim()}`); + } + } + } + expect(offenders).toEqual([]); + }); + + // A topic that does not resolve is the expensive case: the agent has already + // committed to a plan that routes through it before finding out. + it("cites only topics that resolve to an embedded recipe", async () => { + const topics = await embeddedTopics(); + const dangling: string[] = []; + + for (const file of await recipeFiles()) { + const source = await readFile(join(helpDirectory, file), "utf8"); + for (const [index, line] of source.split("\n").entries()) { + // `taskless agent `, however it is punctuated around. + for (const match of line.matchAll(/taskless agent ([a-z][a-z-]*)/g)) { + const topic = match[1]; + if (topic !== undefined && !topics.has(topic)) { + dangling.push(`${file}:${String(index + 1)} cites '${topic}'`); + } + } + } + } + + expect(dangling).toEqual([]); + }); + + // Every recipe is reachable by the name its filename implies. A header that + // disagrees with the filename means `taskless agent ` prints a recipe + // announcing itself as something else. + it("opens each recipe with a header naming its own topic", async () => { + const mismatched: string[] = []; + for (const file of await recipeFiles()) { + const source = await readFile(join(helpDirectory, file), "utf8"); + const topic = file.replace(/(\.anonymous)?\.txt$/, ""); + const firstLine = source.split("\n")[0] ?? ""; + if (!firstLine.startsWith(`# Topic: ${topic}`)) { + mismatched.push(`${file}: ${firstLine}`); + } + } + expect(mismatched).toEqual([]); + }); +}); diff --git a/skills/taskless/SKILL.md b/skills/taskless/SKILL.md index 1a32ef1..6801a97 100644 --- a/skills/taskless/SKILL.md +++ b/skills/taskless/SKILL.md @@ -17,7 +17,7 @@ description: | Also trigger on any request to add/write/create a lint or code rule, including ones that name a specific tool (eslint, ruff, biome, stylelint, ast-grep). Naming a tool ENGAGES this skill's routing flow via - `taskless help route`; it does NOT suppress the skill. + `taskless agent route`; it does NOT suppress the skill. metadata: author: taskless version: 0.10.2 From 8c8627c89bbfb9dc1598f335ddd6f7eca00c70be Mon Sep 17 00:00:00 2001 From: Jakob Heuser Date: Fri, 14 Aug 2026 21:48:30 -0700 Subject: [PATCH 19/19] docs(openspec): archive agent-command-and-vale-authoring MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merges the deltas into specs/ and creates cli-agent-authoring. The archive refused three times, each for a real defect in the deltas rather than a tooling complaint: `cli-help` MODIFIED "Routing recipes name a destination", a header that does not exist because the requirement is a rename of "Routing recipes reference engine selection". MODIFIED matches by header, so a rename has to be a REMOVE plus an ADD. The delta also never removed "The engine-selection topic is registered in the help system", which registers a topic this change deletes. `cli-rule-routing` MODIFIED "Trust tier is not an engine-selection input", which is new, so it belongs under ADDED. Then `cli-help` MODIFIED requirements the parser could not see. The spec carried an orphaned duplicate of "Help text files follow a consistent format" whose `### Requirement:` header and opening code fence had both been lost, so a documentation template parsed as real `## Goal`/`## Preconditions` headings, ending the Requirements section and stranding every requirement below it. The stray closing fence was the tell. Pre-existing on main. Removed the duplicate; the live copy is unchanged. Also brings along the requirements the rename left describing a command that no longer exists: the onboard topic, routing topic registration, anonymous variants, embedded schemas, and both telemetry requirements. The telemetry ones were already wrong before this change — they specify per-topic `help_` events, and the code emits one `cli_help` event with a `topic` property. Corrected to what the code does. Renames two sibling requirements the delta had skipped while renaming their peer: "Existing recipe" and "Remote recipe" now name create-legacy-rule and create-remote-rule. `cli-rules` and `cli-update-engine` still fail --strict; both fail on main and neither is touched here. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Jwc9FFroR3mTZ4hLiSkkX3 --- .../specs/cli-help/spec.md | 63 ----- .../.openspec.yaml | 0 .../design.md | 0 .../iteration-log.md | 0 .../proposal.md | 0 .../resume.md | 0 .../specs/cli-agent-authoring/spec.md | 0 .../specs/cli-help/spec.md | 183 +++++++++++++ .../specs/cli-knowledge-prompts/spec.md | 0 .../specs/cli-rule-routing/spec.md | 24 +- .../specs/cli-vale-rule-engine/spec.md | 0 .../tasks.md | 0 openspec/specs/cli-agent-authoring/spec.md | 104 ++++++++ openspec/specs/cli-help/spec.md | 251 +++++++----------- openspec/specs/cli-knowledge-prompts/spec.md | 37 ++- openspec/specs/cli-rule-routing/spec.md | 178 ++++++------- openspec/specs/cli-vale-rule-engine/spec.md | 41 ++- 17 files changed, 547 insertions(+), 334 deletions(-) delete mode 100644 openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/.openspec.yaml (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/design.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/iteration-log.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/proposal.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/resume.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/specs/cli-agent-authoring/spec.md (100%) create mode 100644 openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-help/spec.md rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/specs/cli-knowledge-prompts/spec.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/specs/cli-rule-routing/spec.md (85%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/specs/cli-vale-rule-engine/spec.md (100%) rename openspec/changes/{agent-command-and-vale-authoring => archive/2026-08-15-agent-command-and-vale-authoring}/tasks.md (100%) create mode 100644 openspec/specs/cli-agent-authoring/spec.md diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md b/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md deleted file mode 100644 index 268167a..0000000 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-help/spec.md +++ /dev/null @@ -1,63 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Help subcommand displays rich help text for commands - -The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic AND an optional `--anonymous` boolean flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching help text file embedded at build time using the following resolution order: - -1. If `--anonymous` is set AND `.anonymous.txt` exists in the embedded map, return that file. -2. Otherwise, return `.txt`. -3. If neither exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index. - -When no positional argument is provided, the subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries. - -The subcommand is named for its reader. It serves agents fetching a procedure, not humans asking for help, and single-token addressing exists so a topic name is a literal string an agent copies rather than a phrase it can reorder or paraphrase. - -#### Scenario: Agent subcommand for a topic returns the recipe - -- **WHEN** a user runs `taskless agent check` -- **THEN** the CLI SHALL print the contents of `check.txt` to stdout - -#### Scenario: Multi-word topic paths are not resolved - -- **WHEN** a user runs `taskless agent rule create` -- **THEN** the CLI SHALL NOT look up `rule-create.txt` by joining the positionals -- **AND** it SHALL exit non-zero rather than guessing a topic - -#### Scenario: Formerly nested topics are addressed by one token - -- **WHEN** a user runs `taskless agent create-rule` -- **THEN** the CLI SHALL look up `create-rule.txt` and print its contents - -#### Scenario: The former command name is gone - -- **WHEN** a user runs `taskless help check` -- **THEN** the CLI SHALL NOT print recipe text for `check` - -### Requirement: Routing recipes name a destination, not a second decision - -The `route` recipe SHALL apply the engine reasoning directly and name a concrete `create-*-rule` topic, rather than referring the reader onward to a topic that selects an engine. No shipped recipe SHALL refer to `engine-selection`, which no longer exists. - -Each `create-*-rule` recipe SHALL instead point back at `route` for a reader who arrived at the wrong one, so recovery costs a re-decision rather than a second copy of the criterion (see "Every authoring recipe opens by orienting the reader"). - -#### Scenario: Route names a destination without a second fetch - -- **WHEN** an agent follows `route` -- **THEN** the recipe SHALL name one `create-*-rule` topic -- **AND** it SHALL NOT require fetching a separate engine-selection topic first to do so - -#### Scenario: Authoring recipes point back rather than re-deciding - -- **WHEN** an agent reads any `create-*-rule` recipe -- **THEN** the recipe SHALL name `route` as where to go if this is the wrong destination -- **AND** it SHALL NOT reference `engine-selection` - -## ADDED Requirements - -### Requirement: Shipped recipes name only commands that exist - -No embedded recipe SHALL contain the string `taskless help`. Recipes cross-reference each other by literal command string, so a stale reference is invisible until an agent runs it and receives nothing. - -#### Scenario: No recipe references the removed command - -- **WHEN** the embedded recipe set is inspected -- **THEN** no recipe SHALL contain `taskless help` diff --git a/openspec/changes/agent-command-and-vale-authoring/.openspec.yaml b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/.openspec.yaml similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/.openspec.yaml rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/.openspec.yaml diff --git a/openspec/changes/agent-command-and-vale-authoring/design.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/design.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/design.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/design.md diff --git a/openspec/changes/agent-command-and-vale-authoring/iteration-log.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/iteration-log.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/iteration-log.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/iteration-log.md diff --git a/openspec/changes/agent-command-and-vale-authoring/proposal.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/proposal.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/proposal.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/proposal.md diff --git a/openspec/changes/agent-command-and-vale-authoring/resume.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/resume.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/resume.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/resume.md diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-agent-authoring/spec.md diff --git a/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-help/spec.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-help/spec.md new file mode 100644 index 0000000..3da108d --- /dev/null +++ b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-help/spec.md @@ -0,0 +1,183 @@ +## MODIFIED Requirements + +### Requirement: Help subcommand displays rich help text for commands + +The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic AND an optional `--anonymous` boolean flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching help text file embedded at build time using the following resolution order: + +1. If `--anonymous` is set AND `.anonymous.txt` exists in the embedded map, return that file. +2. Otherwise, return `.txt`. +3. If neither exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index. + +When no positional argument is provided, the subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries. + +The subcommand is named for its reader. It serves agents fetching a procedure, not humans asking for help, and single-token addressing exists so a topic name is a literal string an agent copies rather than a phrase it can reorder or paraphrase. + +#### Scenario: Agent subcommand for a topic returns the recipe + +- **WHEN** a user runs `taskless agent check` +- **THEN** the CLI SHALL print the contents of `check.txt` to stdout + +#### Scenario: Multi-word topic paths are not resolved + +- **WHEN** a user runs `taskless agent rule create` +- **THEN** the CLI SHALL NOT look up `rule-create.txt` by joining the positionals +- **AND** it SHALL exit non-zero rather than guessing a topic + +#### Scenario: Formerly nested topics are addressed by one token + +- **WHEN** a user runs `taskless agent improve-rule` +- **THEN** the CLI SHALL look up `improve-rule.txt` and print its contents + +#### Scenario: The former command name is gone + +- **WHEN** a user runs `taskless help check` +- **THEN** the CLI SHALL NOT print recipe text for `check` + +### Requirement: onboard topic is registered in the help index + +A help topic `onboard` SHALL be registered. The CLI SHALL embed `packages/cli/src/help/onboard.txt` at build time via the existing `import.meta.glob` mechanism. `taskless agent onboard` SHALL print the contents of `onboard.txt`. The topic SHALL appear in the output of `taskless agent` (the index) with a one-line summary describing it as the post-install rule-discovery flow. + +#### Scenario: The onboard topic returns the recipe + +- **WHEN** a user runs `taskless agent onboard` +- **THEN** the CLI SHALL print the contents of `onboard.txt` to stdout +- **AND** SHALL exit with code 0 + +#### Scenario: Topic index includes onboard + +- **WHEN** a user runs `taskless agent` (no args) +- **THEN** the topic index SHALL include a row for `onboard` +- **AND** the row SHALL describe it as the post-install rule-discovery flow + +### Requirement: help_onboard intent telemetry + +Fetching the `onboard` topic SHALL emit the command's single intent event, `cli_help`, carrying `onboard` as its `topic` property. + +Per-topic event names (`help_onboard` and siblings) are not emitted. One event with a topic property is filterable the same way and does not grow the event vocabulary every time a topic is added or renamed, which this change would otherwise have to do for every rename below. + +#### Scenario: Fetching onboard captures its topic + +- **WHEN** an agent runs `taskless agent onboard` +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `onboard` + +### Requirement: Routing topics are registered in the help system + +The help system SHALL register `route` and each `create-*-rule` recipe as embedded topics, retrievable via `taskless agent ` and listed in the topic index, consistent with the existing topic embedding and format requirements. + +`existing`, `static`, and `remote` are no longer topics. `route` applies the criterion they carried and names a concrete destination, so an agent reaches an authoring recipe in one fetch. + +#### Scenario: Routing topics resolve + +- **WHEN** `taskless agent route` or any `taskless agent create-*-rule` is run +- **THEN** the corresponding recipe text SHALL be returned +- **AND** an unknown-topic error SHALL NOT be raised + +#### Scenario: Removed routing topics do not resolve + +- **WHEN** `taskless agent existing`, `taskless agent static`, or `taskless agent remote` is run +- **THEN** the CLI SHALL exit non-zero +- **AND** it SHALL NOT print recipe text + +#### Scenario: Routing topics appear in the index + +- **WHEN** `taskless agent` (no arguments) is run +- **THEN** the topic index SHALL include `route` and every `create-*-rule` topic + +### Requirement: Routing topics emit intent telemetry + +Fetching a routing recipe SHALL emit the command's single intent event, `cli_help`, carrying the served topic as its `topic` property. + +#### Scenario: Intent is captured for routing recipes + +- **WHEN** the agent fetches `route` or any `create-*-rule` topic +- **THEN** the command SHALL capture a `cli_help` event whose `topic` property is that topic name + +### Requirement: Anonymous variant lookup uses a compile-time map + +The help command SHALL construct, at build time, a Set of topic names that have a corresponding `.anonymous.txt` file. Lookup at runtime SHALL be O(1). The Set SHALL be derived from `import.meta.glob` matching `*.anonymous.txt` in the help directory. + +#### Scenario: Topics with variants are detected at build time + +- **WHEN** the CLI bundle is built +- **AND** a file `improve-rule.anonymous.txt` exists +- **THEN** the embedded variants set SHALL contain `improve-rule` + +#### Scenario: Topics without variants are absent from the map + +- **WHEN** the CLI bundle is built +- **AND** no `check.anonymous.txt` file exists +- **THEN** the embedded variants set SHALL NOT contain `check` +- **AND** `taskless agent check --anonymous` SHALL fall back to `check.txt` + +### Requirement: Embedded JSON schemas are generated via zod-to-json-schema + +For every recipe topic that documents a CLI command accepting `--from `, the corresponding Zod input schema in `packages/cli/src/schemas/` SHALL be converted to JSON Schema and embedded in the recipe's `## Input schema` section as a fenced code block. Generation MAY happen at runtime (small dep, fast) or at build time; runtime is acceptable. + +#### Scenario: The remote authoring recipe embeds its input schema + +- **WHEN** a user runs `taskless agent create-remote-rule` +- **THEN** the output SHALL contain an `## Input schema` section +- **AND** the section SHALL contain a code-fenced JSON Schema block derived from the `rules-create` Zod schema + +#### Scenario: The improve recipe embeds its input schema + +- **WHEN** a user runs `taskless agent improve-rule` +- **THEN** the output SHALL contain an `## Input schema` section with the rule-improve JSON Schema + +### Requirement: Help command emits intent telemetry + +The `agent` command SHALL emit one PostHog event, `cli_help`, on every invocation, carrying a `topic` property: + +- the served topic when a positional resolves to a known topic +- the attempted topic string when it resolves to none +- `(index)` when called with no positional arguments +- the joined positionals when more than one is supplied + +#### Scenario: Topic fetch captures the topic + +- **WHEN** an agent runs `taskless agent create-sg-rule` +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `create-sg-rule` + +#### Scenario: Index fetch captures the index + +- **WHEN** an agent runs `taskless agent` (no args) +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `(index)` + +## ADDED Requirements + +### Requirement: Routing recipes name a destination, not a second decision + +The `route` recipe SHALL apply the engine reasoning directly and name a concrete `create-*-rule` topic, rather than referring the reader onward to a topic that selects an engine. No shipped recipe SHALL refer to `engine-selection`, which no longer exists. + +Each `create-*-rule` recipe SHALL instead point back at `route` for a reader who arrived at the wrong one, so recovery costs a re-decision rather than a second copy of the criterion (see "Every authoring recipe opens by orienting the reader"). + +#### Scenario: Route names a destination without a second fetch + +- **WHEN** an agent follows `route` +- **THEN** the recipe SHALL name one `create-*-rule` topic +- **AND** it SHALL NOT require fetching a separate engine-selection topic first to do so + +#### Scenario: Authoring recipes point back rather than re-deciding + +- **WHEN** an agent reads any `create-*-rule` recipe +- **THEN** the recipe SHALL name `route` as where to go if this is the wrong destination +- **AND** it SHALL NOT reference `engine-selection` + +### Requirement: Shipped recipes name only commands that exist + +No embedded recipe SHALL contain the string `taskless help`. Recipes cross-reference each other by literal command string, so a stale reference is invisible until an agent runs it and receives nothing. + +#### Scenario: No recipe references the removed command + +- **WHEN** the embedded recipe set is inspected +- **THEN** no recipe SHALL contain `taskless help` + +## REMOVED Requirements + +### Requirement: The engine-selection topic is registered in the help system + +**Reason**: The topic no longer exists. Its criterion moved into `route`, which now applies the engine reasoning itself and names a concrete destination, so there is nothing left to register or to fetch. + +### Requirement: Routing recipes reference engine selection + +**Reason**: Replaced by "Routing recipes name a destination, not a second decision". The requirement named `route` and `static` and obliged them to forward to a separate engine-selection topic; `static` is gone, and forwarding is the behavior this change removes. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-knowledge-prompts/spec.md diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md similarity index 85% rename from openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md index a395e83..cdfa94f 100644 --- a/openspec/changes/agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md +++ b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-rule-routing/spec.md @@ -91,6 +91,28 @@ When no engine is clearly indicated, `route` SHALL direct the reader to choose a - **WHEN** an engine is unavailable in the current environment, such as the Vale binary being absent - **THEN** the ambiguity default SHALL NOT name it +### Requirement: Existing recipe authors in the detected linter's dialect + +The CLI SHALL provide a `create-legacy-rule` help recipe that instructs the agent to author a rule in a linter already detected in the repository, expressed in that tool's own dialect. The recipe SHALL direct the agent to source authoring knowledge first from the repository's own existing rules and only then from the agent's own web research. The recipe SHALL NOT embed or rely on a Taskless-maintained catalog of linter rules. + +The recipe is named for the artifact it produces. "Existing" described the repository's state rather than the rule being written, which is not something an agent can address by name. + +#### Scenario: Repo-first knowledge sourcing + +- **WHEN** the agent follows `create-legacy-rule` +- **THEN** it SHALL read the repository's own rules for that linter before consulting any external source + +### Requirement: Remote recipe collects inputs and delegates to the service + +The CLI SHALL provide a `create-remote-rule` help recipe that instructs the agent to gather the inputs required to call the Taskless service and to invoke the existing rule generation backend, which runs the service-side classifier and returns either a static or a runtime rule. The recipe SHALL require authentication and SHALL NOT itself decide static versus runtime. + +#### Scenario: The remote recipe requires authentication + +- **WHEN** the agent follows `create-remote-rule` while logged out +- **THEN** the recipe SHALL direct the agent to `auth` rather than calling the service + +## ADDED Requirements + ### Requirement: Trust tier is not an engine-selection input Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to the engine choice: `sg` and `vale` are both static-tier, and only `runtime` carries those concerns, so trust tier is a distinct axis from which engine can express a rule. @@ -100,8 +122,6 @@ Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to - **WHEN** the reasoning distinguishes `sg` from `vale` - **THEN** it does so on the prose-versus-structure axis, not on any auth, reconcile, or signing property, since both are static-tier -## ADDED Requirements - ### Requirement: Engine reasoning lives in route and in each destination The engine criterion SHALL be stated once, in `route`'s destination table, which is where the comparison between engines is made. It SHALL NOT be stated in a separate chooser topic, and SHALL NOT be restated in the destination recipes. diff --git a/openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/specs/cli-vale-rule-engine/spec.md diff --git a/openspec/changes/agent-command-and-vale-authoring/tasks.md b/openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/tasks.md similarity index 100% rename from openspec/changes/agent-command-and-vale-authoring/tasks.md rename to openspec/changes/archive/2026-08-15-agent-command-and-vale-authoring/tasks.md diff --git a/openspec/specs/cli-agent-authoring/spec.md b/openspec/specs/cli-agent-authoring/spec.md new file mode 100644 index 0000000..ad09bd0 --- /dev/null +++ b/openspec/specs/cli-agent-authoring/spec.md @@ -0,0 +1,104 @@ +# cli-agent-authoring Specification + +## Purpose +TBD - created by archiving change agent-command-and-vale-authoring. Update Purpose after archive. +## Requirements +### Requirement: Every engine a rule can be routed to has an authoring recipe + +The CLI SHALL provide an authoring recipe for each engine `route` can name: `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule`, alongside `create-legacy-rule` for a linter the repository already uses. + +A decision procedure that can produce an answer with no destination is incomplete. Engine selection can conclude `vale` or `runtime`, and before this change neither had a procedure, so an agent that reasoned correctly arrived nowhere. + +#### Scenario: Each engine choice reaches a procedure + +- **WHEN** engine selection concludes `sg`, `vale`, or `runtime` +- **THEN** a recipe exists that authors a rule for that engine + +#### Scenario: A legacy destination exists for repositories with their own linter + +- **WHEN** the repository already runs a linter that can express the rule +- **THEN** `create-legacy-rule` SHALL author it in that tool's own dialect + +### Requirement: The Vale authoring recipe covers rule, scope, and fixtures + +The `create-vale-rule` recipe SHALL instruct the agent to produce three artifacts, and SHALL state that a rule is incomplete without all three: + +1. A Vale style file under `.taskless/vale/rules/.yml`. +2. A section in the committed `.taskless/vale/.vale.ini` scoping which files the rule applies to, enabling it as `rules. = YES`. +3. `pass/` and `fail/` fixture documents under `.taskless/vale/rule-tests//`. + +The recipe SHALL state that the scaffolded config carries no section, so the first rule authored in a project also authors the first scope. + +#### Scenario: Authoring produces all three artifacts + +- **WHEN** the agent follows `create-vale-rule` +- **THEN** it writes the style file, a scoping section enabling the rule, and both fixture buckets + +#### Scenario: The recipe teaches the first section + +- **WHEN** a project's `.vale.ini` has no section yet +- **THEN** the recipe SHALL direct the agent to add one scoped to the files the rule is about, rather than assuming a section exists + +#### Scenario: An unscoped rule is not silently accepted + +- **WHEN** the agent enables a rule without placing it inside a section +- **THEN** the recipe SHALL identify this as incomplete, because Vale ignores a rule assignment outside a section + +### Requirement: Authoring recipes write files rather than invoking a writer + +The `create-*-rule` recipes SHALL instruct the agent to write the rule, its configuration, and its fixtures directly. The CLI SHALL NOT provide a command that generates a Vale style file or edits `.vale.ini` on the agent's behalf. + +This matches how ast-grep rules are authored today: the agent writes the rule and its config entry, and construction belongs to the downstream generator rather than to the CLI. + +#### Scenario: No CLI writer for Vale configuration + +- **WHEN** an agent authors a Vale rule +- **THEN** it edits `.vale.ini` itself +- **AND** the CLI SHALL NOT offer a subcommand that performs that edit + +### Requirement: The runtime authoring recipe is the logged-out path + +The `create-runtime-rule` recipe SHALL explain that runtime rules execute code and therefore require login, reconciliation, and signing, and SHALL state this as a property of executing code rather than of the engine's capability. It SHALL point at `auth` for obtaining access rather than restating the login procedure, which `auth` owns. + +It SHALL NOT forward the agent to another authoring recipe. A logged-in runtime request is routed to `create-remote-rule` by `route`, so this recipe is reached only when the gate is closed and exists to explain that one gate once. + +#### Scenario: The gate is explained where it is encountered + +- **WHEN** an agent follows `create-runtime-rule` +- **THEN** the recipe SHALL state why the runtime tier is gated when the static tiers are not +- **AND** it SHALL refer the reader to `auth` rather than restating how to log in + +#### Scenario: The recipe does not delegate + +- **WHEN** an agent follows `create-runtime-rule` +- **THEN** it SHALL NOT be directed to fetch another authoring recipe to proceed + +### Requirement: Service generation is one recipe + +The CLI SHALL provide a single `create-remote-rule` recipe covering both the client-side boundary of service generation and the procedure itself — enriching the user's description, dispatching to the Taskless service, and reporting the result. + +Split across a boundary statement and a procedure, an agent fetches one only to learn it needs the other, which is the second fetch this change exists to remove. + +#### Scenario: One fetch reaches the whole procedure + +- **WHEN** an agent follows `create-remote-rule` +- **THEN** the recipe SHALL carry both the boundary and the dispatch procedure +- **AND** it SHALL NOT require fetching a second topic to complete the request + +### Requirement: Every authoring recipe opens by orienting the reader + +Each `create-*-rule` recipe SHALL open with a line naming the topic the reader is in, the kinds of rule it helps write, and an instruction to revisit the routing decision if that is not what they need. + +The line SHALL orient, not classify: it states this recipe's own scope and SHALL NOT restate the criterion distinguishing the engines from each other, which `route` holds in one place. An agent that arrived at the wrong recipe — by guessing, by a user naming a topic directly, or because `route` was wrong — should discover it in the first line, where recovery is cheap, rather than after authoring the wrong artifact. + +#### Scenario: A misrouted reader is told how to recover + +- **WHEN** an agent opens any `create-*-rule` recipe +- **THEN** the first lines SHALL name what that recipe helps write +- **AND** SHALL instruct the agent to revisit its routing decision if it needs a different kind of check + +#### Scenario: The orientation is not a second criterion + +- **WHEN** the orientation line is read +- **THEN** it SHALL describe only this recipe's scope, not the comparison between engines + diff --git a/openspec/specs/cli-help/spec.md b/openspec/specs/cli-help/spec.md index ff82a10..46c464c 100644 --- a/openspec/specs/cli-help/spec.md +++ b/openspec/specs/cli-help/spec.md @@ -3,53 +3,39 @@ ## Purpose TBD — Defines the help subcommand for the `@taskless/cli` package, including help text display, embedding, and formatting. - ## Requirements - ### Requirement: Help subcommand displays rich help text for commands -The CLI SHALL support a `help` subcommand that accepts zero or more positional arguments identifying a topic path AND an optional `--anonymous` boolean flag. When positional arguments are provided, the help subcommand SHALL look up a matching help text file embedded at build time using the following resolution order: +The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic AND an optional `--anonymous` boolean flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching help text file embedded at build time using the following resolution order: 1. If `--anonymous` is set AND `.anonymous.txt` exists in the embedded map, return that file. 2. Otherwise, return `.txt`. -3. If neither exists, exit with code 1 and an error message suggesting `taskless help` for the topic index. - -When no positional arguments are provided, the help subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries. +3. If neither exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index. -#### Scenario: Help for a topic returns the recipe +When no positional argument is provided, the subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries. -- **WHEN** a user runs `taskless help check` -- **THEN** the CLI SHALL print the contents of `check.txt` to stdout +The subcommand is named for its reader. It serves agents fetching a procedure, not humans asking for help, and single-token addressing exists so a topic name is a literal string an agent copies rather than a phrase it can reorder or paraphrase. -#### Scenario: Help for a nested topic joins with hyphens +#### Scenario: Agent subcommand for a topic returns the recipe -- **WHEN** a user runs `taskless help rule create` -- **THEN** the CLI SHALL look up `rule-create.txt` and print its contents - -#### Scenario: Help with --anonymous returns the variant when present - -- **WHEN** a user runs `taskless help rule create --anonymous` -- **AND** `rule-create.anonymous.txt` exists in the embedded help map -- **THEN** the CLI SHALL print the contents of `rule-create.anonymous.txt` +- **WHEN** a user runs `taskless agent check` +- **THEN** the CLI SHALL print the contents of `check.txt` to stdout -#### Scenario: Help with --anonymous falls back when no variant exists +#### Scenario: Multi-word topic paths are not resolved -- **WHEN** a user runs `taskless help check --anonymous` -- **AND** no `check.anonymous.txt` exists -- **THEN** the CLI SHALL print the contents of `check.txt` (no error, no warning — anonymous is a no-op for this topic) +- **WHEN** a user runs `taskless agent rule create` +- **THEN** the CLI SHALL NOT look up `rule-create.txt` by joining the positionals +- **AND** it SHALL exit non-zero rather than guessing a topic -#### Scenario: Help with no arguments shows index with human slug and disambiguation table +#### Scenario: Formerly nested topics are addressed by one token -- **WHEN** a user runs `taskless help` -- **THEN** the CLI SHALL print a one-paragraph human-facing slug explaining what the help command does for human vs. agent audiences -- **AND** SHALL print a topic table mapping each topic name to its one-line summary -- **AND** SHALL include a note about the `--anonymous` flag +- **WHEN** a user runs `taskless agent improve-rule` +- **THEN** the CLI SHALL look up `improve-rule.txt` and print its contents -#### Scenario: Help for an unknown topic exits with error +#### Scenario: The former command name is gone -- **WHEN** a user runs `taskless help nonexistent` -- **THEN** the CLI SHALL print an error message indicating the topic is not recognized -- **AND** exit with code 1 +- **WHEN** a user runs `taskless help check` +- **THEN** the CLI SHALL NOT print recipe text for `check` ### Requirement: Help text files are embedded at build time @@ -120,140 +106,62 @@ Recipe authors SHALL escape any literal `%` character in recipe content as `%%` ### Requirement: onboard topic is registered in the help index -A new help topic `onboard` SHALL be registered. The CLI SHALL embed `packages/cli/src/help/onboard.txt` at build time via the existing `import.meta.glob` mechanism. `taskless help onboard` SHALL print the contents of `onboard.txt`. The topic SHALL appear in the output of `taskless help` (the index) with a one-line summary describing it as the post-install rule-discovery flow. +A help topic `onboard` SHALL be registered. The CLI SHALL embed `packages/cli/src/help/onboard.txt` at build time via the existing `import.meta.glob` mechanism. `taskless agent onboard` SHALL print the contents of `onboard.txt`. The topic SHALL appear in the output of `taskless agent` (the index) with a one-line summary describing it as the post-install rule-discovery flow. -#### Scenario: Help for onboard returns the recipe +#### Scenario: The onboard topic returns the recipe -- **WHEN** a user runs `taskless help onboard` +- **WHEN** a user runs `taskless agent onboard` - **THEN** the CLI SHALL print the contents of `onboard.txt` to stdout - **AND** SHALL exit with code 0 -#### Scenario: Help index includes onboard +#### Scenario: Topic index includes onboard -- **WHEN** a user runs `taskless help` (no args) -- **THEN** the topic table SHALL include a row for `onboard` +- **WHEN** a user runs `taskless agent` (no args) +- **THEN** the topic index SHALL include a row for `onboard` - **AND** the row SHALL describe it as the post-install rule-discovery flow -#### Scenario: Onboard recipe is embedded at build time - -- **WHEN** the CLI bundle is built -- **THEN** `import.meta.glob` matching the help directory SHALL include `onboard.txt` -- **AND** the recipe SHALL be available at runtime without filesystem access - -#### Scenario: Help onboard with --anonymous falls back - -- **WHEN** a user runs `taskless help onboard --anonymous` -- **AND** no `onboard.anonymous.txt` exists -- **THEN** the CLI SHALL print the contents of `onboard.txt` (anonymous is a no-op for this topic) - ### Requirement: help_onboard intent telemetry -The help command's existing intent-telemetry requirement SHALL extend naturally to the new topic: invocations of `taskless help onboard` SHALL emit a `help_onboard` PostHog event, consistent with the `help_` pattern. +Fetching the `onboard` topic SHALL emit the command's single intent event, `cli_help`, carrying `onboard` as its `topic` property. + +Per-topic event names (`help_onboard` and siblings) are not emitted. One event with a topic property is filterable the same way and does not grow the event vocabulary every time a topic is added or renamed, which this change would otherwise have to do for every rename below. -#### Scenario: Help onboard emits help_onboard +#### Scenario: Fetching onboard captures its topic -- **WHEN** an agent runs `taskless help onboard` -- **THEN** PostHog SHALL receive a `help_onboard` event +- **WHEN** an agent runs `taskless agent onboard` +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `onboard` ### Requirement: Routing topics are registered in the help system -The help system SHALL register the routing recipes `route`, `existing`, `static`, -and `remote` as embedded help topics, retrievable via `taskless help ` and -listed in the help index, consistent with the existing topic embedding and format -requirements. +The help system SHALL register `route` and each `create-*-rule` recipe as embedded topics, retrievable via `taskless agent ` and listed in the topic index, consistent with the existing topic embedding and format requirements. + +`existing`, `static`, and `remote` are no longer topics. `route` applies the criterion they carried and names a concrete destination, so an agent reaches an authoring recipe in one fetch. #### Scenario: Routing topics resolve -- **WHEN** `taskless help route`, `taskless help existing`, - `taskless help static`, or `taskless help remote` is run +- **WHEN** `taskless agent route` or any `taskless agent create-*-rule` is run - **THEN** the corresponding recipe text SHALL be returned -- **AND** an unknown-topic error SHALL NOT be raised for any of the four - -#### Scenario: Routing topics appear in the index - -- **WHEN** `taskless help` (no arguments) is run -- **THEN** the topic index SHALL include the routing topics so an agent can - discover the authoring front door - -### Requirement: Routing topics emit intent telemetry - -Fetching a routing recipe SHALL emit a per-topic intent telemetry event, -consistent with the existing `help_` telemetry convention. - -#### Scenario: Help topic intent is captured for routing recipes - -- **WHEN** the agent fetches `route`, `existing`, `static`, or `remote` -- **THEN** the help command SHALL capture the corresponding `help_` intent - event with the topic name - -### Requirement: The engine-selection topic is registered in the help system - -The help system SHALL register the engine-selection recipe as an embedded help topic, retrievable via `taskless help ` and listed in the help index, consistent with the existing topic embedding and format requirements. - -#### Scenario: Engine-selection topic resolves - -- **WHEN** `taskless help` is run for the engine-selection topic -- **THEN** the recipe text SHALL be returned and an unknown-topic error SHALL NOT be raised - -#### Scenario: Engine-selection topic appears in the index - -- **WHEN** `taskless help` is run with no arguments -- **THEN** the topic index SHALL include the engine-selection topic so an agent can discover it - -### Requirement: Routing recipes reference engine selection - -The `route` and `static` recipes SHALL reference the engine-selection topic so an agent following the local authoring flow applies the same engine test the service applies, rather than assuming ast-grep. - -#### Scenario: Local flow reaches engine selection +- **AND** an unknown-topic error SHALL NOT be raised -- **WHEN** an agent follows `route` to a destination that authors a Taskless rule -- **THEN** the recipe directs it to the engine-selection topic before the rule is authored +#### Scenario: Removed routing topics do not resolve -## Goal +- **WHEN** `taskless agent existing`, `taskless agent static`, or `taskless agent remote` is run +- **THEN** the CLI SHALL exit non-zero +- **AND** it SHALL NOT print recipe text - - -## Preconditions - - - -## Steps - - - -## Input schema - - - -## Errors - - - -## See Also - - -``` - -The header line SHALL include the CLI version (interpolated at build time) and a topic version integer maintained by the recipe author and bumped when the recipe changes meaningfully. - -#### Scenario: Recipe contains all template sections +#### Scenario: Routing topics appear in the index -- **WHEN** any `.txt` file is read -- **THEN** it SHALL begin with the `# Topic: (CLI v / topic v)` header -- **AND** SHALL contain `## Goal`, `## Preconditions`, `## Steps`, `## Errors`, and `## See Also` sections in that order +- **WHEN** `taskless agent` (no arguments) is run +- **THEN** the topic index SHALL include `route` and every `create-*-rule` topic -#### Scenario: Recipe with --from input includes JSON schema +### Requirement: Routing topics emit intent telemetry -- **WHEN** a topic recipe documents a CLI invocation that uses `--from ` -- **THEN** the recipe SHALL contain an `## Input schema` section with a code-fenced JSON Schema block -- **AND** the JSON Schema SHALL be derived from the corresponding Zod schema in `packages/cli/src/schemas/` +Fetching a routing recipe SHALL emit the command's single intent event, `cli_help`, carrying the served topic as its `topic` property. -#### Scenario: Header version reflects build-time CLI version +#### Scenario: Intent is captured for routing recipes -- **WHEN** the CLI bundle is built -- **THEN** the recipe header's CLI version SHALL be interpolated at build time from `packages/cli/package.json` -- **AND** SHALL match the version reported by `taskless info` +- **WHEN** the agent fetches `route` or any `create-*-rule` topic +- **THEN** the command SHALL capture a `cli_help` event whose `topic` property is that topic name ### Requirement: Anonymous variant lookup uses a compile-time map @@ -262,47 +170,74 @@ The help command SHALL construct, at build time, a Set of topic names that have #### Scenario: Topics with variants are detected at build time - **WHEN** the CLI bundle is built -- **AND** files `rule-create.anonymous.txt` and `rule-improve.anonymous.txt` exist -- **THEN** the embedded variants set SHALL contain `rule-create` and `rule-improve` +- **AND** a file `improve-rule.anonymous.txt` exists +- **THEN** the embedded variants set SHALL contain `improve-rule` #### Scenario: Topics without variants are absent from the map - **WHEN** the CLI bundle is built - **AND** no `check.anonymous.txt` file exists - **THEN** the embedded variants set SHALL NOT contain `check` -- **AND** `taskless help check --anonymous` SHALL fall back to `check.txt` +- **AND** `taskless agent check --anonymous` SHALL fall back to `check.txt` ### Requirement: Embedded JSON schemas are generated via zod-to-json-schema -For every recipe topic that documents a CLI command accepting `--from `, the corresponding Zod input schema in `packages/cli/src/schemas/` SHALL be converted to JSON Schema via `zod-to-json-schema` and embedded in the recipe's `## Input schema` section as a fenced code block. Generation MAY happen at runtime (small dep, fast) or at build time; runtime is acceptable. +For every recipe topic that documents a CLI command accepting `--from `, the corresponding Zod input schema in `packages/cli/src/schemas/` SHALL be converted to JSON Schema and embedded in the recipe's `## Input schema` section as a fenced code block. Generation MAY happen at runtime (small dep, fast) or at build time; runtime is acceptable. -#### Scenario: rule create recipe embeds input schema +#### Scenario: The remote authoring recipe embeds its input schema -- **WHEN** a user runs `taskless help rule create` +- **WHEN** a user runs `taskless agent create-remote-rule` - **THEN** the output SHALL contain an `## Input schema` section -- **AND** the section SHALL contain a code-fenced JSON Schema block derived from the `rules-create` Zod schema (or the renamed `rule-create` schema) +- **AND** the section SHALL contain a code-fenced JSON Schema block derived from the `rules-create` Zod schema -#### Scenario: rule improve recipe embeds input schema +#### Scenario: The improve recipe embeds its input schema -- **WHEN** a user runs `taskless help rule improve` +- **WHEN** a user runs `taskless agent improve-rule` - **THEN** the output SHALL contain an `## Input schema` section with the rule-improve JSON Schema ### Requirement: Help command emits intent telemetry -The help command SHALL emit a PostHog event on every invocation: +The `agent` command SHALL emit one PostHog event, `cli_help`, on every invocation, carrying a `topic` property: + +- the served topic when a positional resolves to a known topic +- the attempted topic string when it resolves to none +- `(index)` when called with no positional arguments +- the joined positionals when more than one is supplied + +#### Scenario: Topic fetch captures the topic + +- **WHEN** an agent runs `taskless agent create-sg-rule` +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `create-sg-rule` + +#### Scenario: Index fetch captures the index + +- **WHEN** an agent runs `taskless agent` (no args) +- **THEN** PostHog SHALL receive a `cli_help` event whose `topic` property is `(index)` + +### Requirement: Routing recipes name a destination, not a second decision + +The `route` recipe SHALL apply the engine reasoning directly and name a concrete `create-*-rule` topic, rather than referring the reader onward to a topic that selects an engine. No shipped recipe SHALL refer to `engine-selection`, which no longer exists. + +Each `create-*-rule` recipe SHALL instead point back at `route` for a reader who arrived at the wrong one, so recovery costs a re-decision rather than a second copy of the criterion (see "Every authoring recipe opens by orienting the reader"). + +#### Scenario: Route names a destination without a second fetch + +- **WHEN** an agent follows `route` +- **THEN** the recipe SHALL name one `create-*-rule` topic +- **AND** it SHALL NOT require fetching a separate engine-selection topic first to do so + +#### Scenario: Authoring recipes point back rather than re-deciding -- `help_` (e.g. `help_rule_create`, `help_check`, `help_auth`) when called with positional arguments resolving to a known topic -- `help_index` when called with no positional arguments -- `help_unknown` (with the attempted topic as a property) when called with positional arguments resolving to no topic +- **WHEN** an agent reads any `create-*-rule` recipe +- **THEN** the recipe SHALL name `route` as where to go if this is the wrong destination +- **AND** it SHALL NOT reference `engine-selection` -These events SHALL replace the previous `cli_help_` events in a single hard rename. +### Requirement: Shipped recipes name only commands that exist -#### Scenario: Topic fetch emits intent event +No embedded recipe SHALL contain the string `taskless help`. Recipes cross-reference each other by literal command string, so a stale reference is invisible until an agent runs it and receives nothing. -- **WHEN** an agent runs `taskless help rule create` -- **THEN** PostHog SHALL receive a `help_rule_create` event +#### Scenario: No recipe references the removed command -#### Scenario: Index fetch emits help_index +- **WHEN** the embedded recipe set is inspected +- **THEN** no recipe SHALL contain `taskless help` -- **WHEN** an agent runs `taskless help` (no args) -- **THEN** PostHog SHALL receive a `help_index` event diff --git a/openspec/specs/cli-knowledge-prompts/spec.md b/openspec/specs/cli-knowledge-prompts/spec.md index 4e96764..335cf45 100644 --- a/openspec/specs/cli-knowledge-prompts/spec.md +++ b/openspec/specs/cli-knowledge-prompts/spec.md @@ -15,9 +15,7 @@ cannot drift into giving different guidance. The export carries no CLI runtime, so a Worker can import it without dragging in the command tree, and topic membership is an explicit hand-maintained list so a new recipe file cannot silently become public API. - ## Requirements - ### Requirement: The package exposes knowledge prompts via a dedicated import The package SHALL expose its knowledge prompts (the `help/*.txt` recipes) through a subpath export `@taskless/cli/prompts`, built into `dist` and listed in `files`, so consumers can import them without invoking the CLI. @@ -108,17 +106,23 @@ Where a `.anonymous.txt` variant exists, the export SHALL make it retriev ### Requirement: Topic names and accessor shape are stable public API -The set of `PromptTopic` names, the `getPrompt`/`PROMPTS` shape, and the existing fields of `PromptOptions` SHALL be treated as public API under semver; recipe _text_ MAY change within a major version. +The set of `PromptTopic` names, the `getPrompt`/`PROMPTS` shape, and the existing fields of `PromptOptions` SHALL be treated as public API; recipe _text_ MAY change freely. + +The package is pre-1.0, so a backwards-incompatible change to that surface SHALL be released as a **MINOR** bump. This is what the leading zero means, and it applies to renaming a topic, removing one, or changing the accessor signature. + +What the requirement actually protects is not the version number but the notice. `TOPICS` is consumed across a deploy boundary, so a downstream consumer breaks when it upgrades rather than when this package builds, and the version alone cannot warn anyone. A breaking change SHALL therefore name the removed or renamed topics explicitly in its changeset. -#### Scenario: Removing a topic is a breaking change +#### Scenario: Renaming or removing a topic - **WHEN** a topic is removed or renamed, or the accessor signature changes -- **THEN** it SHALL be released as a major version bump; a text edit SHALL NOT +- **THEN** it SHALL be released as a MINOR bump +- **AND** the changeset SHALL name the removed or renamed topics +- **AND** a recipe text edit SHALL require neither -#### Scenario: Adding an option is not a breaking change +#### Scenario: Adding an option - **WHEN** a new optional field is added to `PromptOptions` -- **THEN** it SHALL NOT require a major version bump, since existing call sites keep their behavior +- **THEN** it SHALL NOT require more than a PATCH bump, since existing call sites keep their behavior ### Requirement: Topic membership is explicit and verified against the recipe files @@ -140,3 +144,22 @@ An automated check SHALL assert that the set of canonical `help/*.txt` topics on - **WHEN** a recipe file is listed as internal - **THEN** the check SHALL pass and the topic SHALL NOT be a member of `PromptTopic` + +### Requirement: Exported topics cover every engine a rule can be routed to + +`TOPICS` SHALL export the authoring recipe for each engine — `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule`. + +A consumer that can decide a rule belongs to an engine must be able to reach the procedure for authoring one. Exporting a chooser without its destinations reproduces, for the platform generator, the dead end this change removes from the CLI. + +`engine-selection` leaves the export because it stops existing: the criterion it carried now lives in `route`, stated once. `route` is not exported here — it still contains local mechanics a Worker cannot run — so until it is, a consumer gets each destination's own scope from these three and adjudicates a genuinely ambiguous call itself. + +#### Scenario: Every engine's authoring path is reachable from the export + +- **WHEN** a consumer imports `TOPICS` +- **THEN** it SHALL contain `create-sg-rule`, `create-vale-rule`, and `create-runtime-rule` + +#### Scenario: The exported set follows the rename + +- **WHEN** a consumer imports `TOPICS` +- **THEN** it SHALL NOT contain `static` or `engine-selection`, neither of which names a recipe any more + diff --git a/openspec/specs/cli-rule-routing/spec.md b/openspec/specs/cli-rule-routing/spec.md index cebcbd9..bb23c2c 100644 --- a/openspec/specs/cli-rule-routing/spec.md +++ b/openspec/specs/cli-rule-routing/spec.md @@ -3,16 +3,14 @@ ## Purpose TBD - created by archiving change local-rule-routing. Update Purpose after archive. - ## Requirements - ### Requirement: Route is the local authoring classifier -The CLI SHALL provide a `route` help recipe that instructs the agent to classify -a rule-authoring request into one of three destinations — `existing`, `static`, -or `remote` — using `taskless detect --json` signals plus the user's intent. The -`route` recipe SHALL be biased to stay local: it SHALL prefer `existing` or -`static` and SHALL treat `remote` as the escalation of last resort. +The CLI SHALL provide a `route` help recipe that instructs the agent to classify a rule-authoring request into one of five destinations — `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, or `create-remote-rule` — using `taskless detect --json` signals plus the user's intent. The `route` recipe SHALL read the user's login state before dispatching, since it determines which destinations are reachable. It SHALL remain biased to stay local: local authoring that works SHALL NOT be abandoned for the service. + +`route` SHALL decide the engine as part of this classification rather than deferring it to a separate topic. There is one decision, made from one reading of the evidence: whether a rule is expressible locally and which engine can express it are answered from the same signals, so splitting them costs a second fetch and a handoff without adding information. + +Each destination SHALL be a topic an agent can fetch by name, so classifying produces a command to run rather than a category to interpret. #### Scenario: Route fetches detection before classifying @@ -20,12 +18,40 @@ or `remote` — using `taskless detect --json` signals plus the user's intent. T - **THEN** the recipe SHALL direct the agent to run `taskless detect --json` and use its signals as input to the classification -#### Scenario: Route classifies into one of three destinations +#### Scenario: Route classifies into one of five destinations - **WHEN** the agent follows `route` -- **THEN** it SHALL select exactly one of `existing`, `static`, or `remote` +- **THEN** it SHALL select exactly one of `create-legacy-rule`, `create-sg-rule`, `create-vale-rule`, `create-runtime-rule`, or `create-remote-rule` - **AND** it SHALL fetch the corresponding recipe to perform the authoring +#### Scenario: Every destination resolves to a recipe + +- **WHEN** any destination `route` can name is fetched +- **THEN** a recipe of that exact name SHALL exist + +#### Scenario: Service generation is offered only where it is a choice + +- **WHEN** the rule is expressible locally AND the user is logged in +- **THEN** `route` MAY offer `create-remote-rule` as an alternative and ask the user +- **AND WHEN** the user is not logged in, or the rule is not expressible locally +- **THEN** `route` SHALL NOT pose service generation as a choice, because it is not one + +#### Scenario: A logged-in runtime request routes straight to the service + +- **WHEN** the rule requires the runtime engine AND the user is logged in +- **THEN** `route` SHALL name `create-remote-rule` +- **AND** no recipe SHALL forward the agent from one destination to another + +#### Scenario: A logged-out runtime request reaches the explanation + +- **WHEN** the rule requires the runtime engine AND the user is not logged in +- **THEN** `route` SHALL name `create-runtime-rule` + +#### Scenario: The engine is decided without a second fetch + +- **WHEN** the agent follows `route` +- **THEN** it SHALL arrive at an engine-specific recipe without fetching a separate engine-selection topic + ### Requirement: Route states reasoning before naming a destination The `route` recipe SHALL require the agent to write an explicit rationale before @@ -135,141 +161,89 @@ a default. ### Requirement: Existing recipe authors in the detected linter's dialect -The CLI SHALL provide an `existing` help recipe that instructs the agent to -author a rule in a linter already detected in the repository, expressed in that -tool's own dialect. The recipe SHALL direct the agent to source authoring -knowledge first from the repository's own existing rules and only then from the -agent's own web research. The recipe SHALL NOT embed or rely on a Taskless- -maintained catalog of linter rules. - -#### Scenario: Repo-first knowledge sourcing +The CLI SHALL provide a `create-legacy-rule` help recipe that instructs the agent to author a rule in a linter already detected in the repository, expressed in that tool's own dialect. The recipe SHALL direct the agent to source authoring knowledge first from the repository's own existing rules and only then from the agent's own web research. The recipe SHALL NOT embed or rely on a Taskless-maintained catalog of linter rules. -- **WHEN** the agent follows `existing` for a detected linter -- **THEN** it SHALL first mine the repository's existing rules of that kind for - house style -- **AND** SHALL fall back to web research (WebFetch/WebSearch) only when the - repository signal is insufficient +The recipe is named for the artifact it produces. "Existing" described the repository's state rather than the rule being written, which is not something an agent can address by name. -#### Scenario: Existing path is author-only +#### Scenario: Repo-first knowledge sourcing -- **WHEN** the agent authors a rule via `existing` -- **THEN** the recipe SHALL make clear the user's own toolchain runs the rule and - that `taskless check` does not execute the external linter +- **WHEN** the agent follows `create-legacy-rule` +- **THEN** it SHALL read the repository's own rules for that linter before consulting any external source ### Requirement: Static recipe authors a verified local ast-grep rule -The CLI SHALL provide a `static` help recipe that instructs the agent to author a +The CLI SHALL provide a `create-sg-rule` help recipe that instructs the agent to author a local ast-grep rule on-device, without calling the Taskless service, and to verify it against the user's success and failure cases before reporting success. The recipe SHALL produce the canonical on-disk rule shape and paths used by remote generation so that `check`, `improve`, and `verify` see a single dialect. +The recipe SHALL be named for the artifact it produces rather than for a trust tier. "Static" describes when a rule runs, which is a different axis from which engine enforces it, and naming the ast-grep authoring path after the tier taught the conflation that engine selection exists to correct. + #### Scenario: Local authoring without the service -- **WHEN** the agent follows `static` +- **WHEN** the agent follows `create-sg-rule` - **THEN** it SHALL write the rule on-device without requiring login or the Taskless API -#### Scenario: Verification gates success - -- **WHEN** the agent authors a static rule -- **THEN** it SHALL verify the rule against the provided success/failure cases - before reporting the rule as complete - -#### Scenario: Canonical output shape - -- **WHEN** the agent writes a static rule to disk -- **THEN** the files, paths, and shape SHALL match those produced by remote - generation - ### Requirement: Remote recipe collects inputs and delegates to the service -The CLI SHALL provide a `remote` help recipe that instructs the agent to gather -the inputs required to call the Taskless service and to invoke the existing rule -generation backend, which runs the service-side classifier and returns either a -static or a runtime rule. The `remote` recipe SHALL require authentication and -SHALL NOT itself decide static versus runtime. - -#### Scenario: Remote requires authentication - -- **WHEN** the agent follows `remote` while logged out -- **THEN** the recipe SHALL direct the agent to the authentication flow before - submitting the request - -#### Scenario: Static-versus-runtime is decided by the service +The CLI SHALL provide a `create-remote-rule` help recipe that instructs the agent to gather the inputs required to call the Taskless service and to invoke the existing rule generation backend, which runs the service-side classifier and returns either a static or a runtime rule. The recipe SHALL require authentication and SHALL NOT itself decide static versus runtime. -- **WHEN** the agent submits an authored request via `remote` -- **THEN** the recipe SHALL rely on the service to classify static versus runtime -- **AND** SHALL NOT make that determination locally +#### Scenario: The remote recipe requires authentication -#### Scenario: Remote output matches local on-disk shape +- **WHEN** the agent follows `create-remote-rule` while logged out +- **THEN** the recipe SHALL direct the agent to `auth` rather than calling the service -- **WHEN** the service returns a generated rule via `remote` -- **THEN** the written files and paths SHALL match the shape produced by the - local `static` path - -### Requirement: An engine-selection topic states which engine can enforce a rule - -The CLI SHALL provide a knowledge topic that decides, for a requested rule, **which engine can enforce it** — `sg`, `vale`, or `runtime` — valued as the engine's on-disk directory name. The topic SHALL define each engine by the information a rule fundamentally needs: - -- **`sg`** — expressible as a pattern over a single file's syntax tree, including correlation between constructs within that same file via relational operators. -- **`vale`** — the target is prose or markup content rather than code structure. -- **`runtime`** — needs information no single file's syntax tree contains: cross-file consistency, import or call graph, comparison against a non-code file, file metadata, or values requiring normalization a static pattern cannot express. +### Requirement: Available code context outranks the phrasing of the request -The topic SHALL instruct that the decision follow from what the rule fundamentally needs rather than how the request was phrased, and that the reasoning be stated before the engine is named. +Where code or diff context is available, `route` SHALL weigh the concrete syntactic form present in the repository above the wording of the request, since the same request routes differently depending on the form the code actually takes. -#### Scenario: Engine named for a single-file structural rule +This bound the standalone engine-selection topic. That topic is gone, but the reasoning is not — it now binds the place the decision is actually made. -- **WHEN** the topic is applied to a request expressible as a pattern over one file's syntax tree -- **THEN** it selects `sg` +#### Scenario: Concrete form changes the engine -#### Scenario: Engine named for a prose rule +- **WHEN** a rule is statically correlatable in the form the repository actually contains +- **THEN** `route` selects `create-sg-rule` +- **AND WHEN** the equivalent rule requires normalizing a captured value to match a declaration elsewhere +- **THEN** it selects a runtime destination, despite an identically phrased request -- **WHEN** the topic is applied to a request targeting prose or markup content -- **THEN** it selects `vale` +### Requirement: Ambiguity resolves to an engine known to be available -#### Scenario: Engine named for a cross-file rule +When no engine is clearly indicated, `route` SHALL direct the reader to choose an engine whose availability can be asserted in the situation at hand, and to give that availability as the reason for the call. It SHALL NOT name a fixed fallback engine. Both `sg` and `vale` ship as platform binaries, so either can be the missing one on an unsupported architecture or where an install was blocked; server-side the constraint is different again, `sg` being the only ungated route. A named default is wrong in whichever of those situations it failed to anticipate, which is why the requirement is stated as a property rather than as a fact about any one engine. -- **WHEN** the topic is applied to a request requiring information beyond a single file's syntax tree -- **THEN** it selects `runtime` +#### Scenario: Ambiguous request resolves to an assertably available engine -### Requirement: Engine selection is a separate axis from authoring destination +- **WHEN** the available context does not disambiguate which engine can enforce a rule +- **THEN** `route` selects an engine whose availability it can assert, and states that availability as the reasoning that made the call close -The engine-selection topic SHALL decide only which engine enforces a rule, and SHALL NOT decide where the rule is authored — that remains the `route` topic's concern. Locally the two compose, `route` first and engine selection second. +#### Scenario: The default is never an unavailable engine -The topic SHALL NOT describe login, reconciliation, or signing as inputs to the engine choice: `sg` and `vale` are both static-tier, and only `runtime` carries those concerns, so trust tier is a distinct axis from engine selection. +- **WHEN** an engine is unavailable in the current environment, such as the Vale binary being absent +- **THEN** the ambiguity default SHALL NOT name it -#### Scenario: Topic stays clear of authoring destination +### Requirement: Trust tier is not an engine-selection input -- **WHEN** the engine-selection topic is applied -- **THEN** it names an engine and does not select among `existing`, `static`, or `remote` authoring destinations +Engine reasoning SHALL NOT treat login, reconciliation, or signing as inputs to the engine choice: `sg` and `vale` are both static-tier, and only `runtime` carries those concerns, so trust tier is a distinct axis from which engine can express a rule. #### Scenario: Trust tier is not an engine-selection input -- **WHEN** the topic distinguishes `sg` from `vale` +- **WHEN** the reasoning distinguishes `sg` from `vale` - **THEN** it does so on the prose-versus-structure axis, not on any auth, reconcile, or signing property, since both are static-tier -### Requirement: Available code context outranks the phrasing of the request - -Where code or diff context is available, the engine-selection topic SHALL weigh the concrete syntactic form present in the repository above the wording of the request, since the same request routes differently depending on the form the code actually takes. - -#### Scenario: Concrete form changes the engine +### Requirement: Engine reasoning lives in route and in each destination -- **WHEN** a rule is statically correlatable in the form the repository actually contains -- **THEN** the topic selects `sg` -- **AND WHEN** the equivalent rule requires normalizing a captured value to match a declaration elsewhere -- **THEN** it selects `runtime`, despite an identically phrased request +The engine criterion SHALL be stated once, in `route`'s destination table, which is where the comparison between engines is made. It SHALL NOT be stated in a separate chooser topic, and SHALL NOT be restated in the destination recipes. -### Requirement: Ambiguity resolves to an engine known to be available +One statement is the point. A criterion copied into each destination is five copies of one test, and the first edit to any of them is a divergence nobody notices — the drift this merge exists to remove, reappearing one level down. Destinations orient the reader to their own scope instead, which needs nothing about the other engines. -When no engine is clearly indicated, the engine-selection topic SHALL direct the reader to choose an engine whose availability can be asserted in the situation at hand, and to give that availability as the reason for the call. The topic SHALL NOT name a fixed fallback engine. Both `sg` and `vale` ship as platform binaries, so either can be the missing one on an unsupported architecture or where an install was blocked; server-side the constraint is different again, `sg` being the only ungated route. A named default is wrong in whichever of those situations it failed to anticipate, which is why the requirement is stated as a property rather than as a fact about any one engine. +#### Scenario: The comparison lives in one place -#### Scenario: Ambiguous request resolves to an assertably available engine +- **WHEN** the embedded recipe set is inspected +- **THEN** exactly one recipe SHALL state the criterion distinguishing the engines from each other -- **WHEN** the available context does not disambiguate which engine can enforce a rule -- **THEN** the topic selects an engine whose availability it can assert, and states that availability as the reasoning that made the call close +#### Scenario: No separate chooser topic exists -#### Scenario: The default is never an unavailable engine +- **WHEN** the embedded recipe set is inspected +- **THEN** there SHALL be no topic whose only purpose is selecting among engines -- **WHEN** an engine is unavailable in the current environment, such as the Vale binary being absent -- **THEN** the ambiguity default SHALL NOT name it diff --git a/openspec/specs/cli-vale-rule-engine/spec.md b/openspec/specs/cli-vale-rule-engine/spec.md index f87026c..0c67b28 100644 --- a/openspec/specs/cli-vale-rule-engine/spec.md +++ b/openspec/specs/cli-vale-rule-engine/spec.md @@ -3,9 +3,7 @@ ## Purpose TBD - created by archiving change add-vale-rule-engine. Update Purpose after archive. - ## Requirements - ### Requirement: Vale runs in the static tier without reconciliation or signing The system SHALL treat Vale as a static-tier engine — always run, with no server reconciliation or signature verification. Vale's `script` checks execute in a sandbox that exposes only pure-computation modules (`text`/`math`/`fmt`) with no host access, so a Vale rule is inert data equivalent in trust to a static ast-grep rule. @@ -122,3 +120,42 @@ Any Taskless-owned breadcrumb the system records in `.vale.ini` SHALL use a `tsk - **WHEN** a rule's scoping spans several matchers each tagged `tskl) rule = no-simply` - **THEN** tooling can find every matcher owned by `no-simply` by its `tskl) rule` id rather than by glob + +### Requirement: The scaffolded Vale config carries no section + +The `.vale.ini` written when a project is scaffolded SHALL contain `StylesPath` and `MinAlertLevel` and no section. A project therefore lints nothing with Vale until someone scopes something deliberately. + +An unscoped `[*]` applies every enabled rule to every file the walk reaches, which makes the default the most aggressive scope available rather than the narrowest. Scope is the author's decision, and the scaffold SHALL NOT make it on their behalf. + +#### Scenario: A freshly scaffolded project reports nothing + +- **WHEN** `check` runs against a scaffolded project with a rule file present and no section added +- **THEN** Vale SHALL report no findings +- **AND** the run SHALL NOT be reported as an engine failure + +#### Scenario: Scope is added by the author + +- **WHEN** an author scopes a rule by adding a section +- **THEN** only files matching that section SHALL be subject to it + +### Requirement: Vale diagnostics on a successful run are surfaced as notices + +When Vale exits zero and writes to stderr, the CLI SHALL surface that output as a notice on the check result. A notice SHALL NOT affect the exit code. + +This is a precondition of the section-less scaffold rather than an independent improvement. With no section to copy, the likely first edit is a rule assignment at the top level of the file, which Vale reports as ignoring — on stderr, with a zero exit and a well-formed empty result. Discarding that output leaves the author with a rule that verifies, runs, and reports nothing, which is the silent-disable failure this engine's design exists to prevent. + +#### Scenario: An ignored rule assignment reaches the user + +- **WHEN** `.vale.ini` enables a rule outside any section and `check` runs +- **THEN** the CLI SHALL surface Vale's diagnostic that the assignment was ignored + +#### Scenario: A diagnostic does not fail the check + +- **WHEN** Vale exits zero, writes a diagnostic to stderr, and reports no findings +- **THEN** the check SHALL exit zero + +#### Scenario: Silence stays silent + +- **WHEN** Vale exits zero and writes nothing to stderr +- **THEN** the CLI SHALL add no notice +