feat: add cursor-first pagination (--limit/--continue) - #118
jpage-godaddy wants to merge 14 commits into
Conversation
Implements the design in docs/proposals/cursor-first-pagination.md: CommandSpec::with_cursor/CursorConfig register --limit/--continue as a new, parallel pagination mechanism alongside the existing offset-based CommandSpec::with_pagination/PaginationConfig (--limit/--offset), which is untouched and remains fully supported. Unlike offset pagination, the engine never slices or measures a cursor itself — a cursor is backend-opaque, so only the handler that talked to the backend can supply the next resume token. Handlers report what they learned via CommandResult::with_cursor(CursorContinuation), which surfaces as a new envelope.cursor field (CursorMeta) and an automatic next_actions "next page" suggestion, mirroring the existing offset pagination machinery end to end: flag registration, middleware state, envelope construction, human-output rendering (table footer merge and standalone summary), and docs. CursorContinuation::with_limit lets a handler report an effective page size that differs from the parsed --limit (e.g. one it derived from the --continue token itself, so a caller can resume with --continue alone without repeating --limit) — this also signals the engine to omit --limit from the auto-generated next-page command, since the token is then self-sufficient about size. A command opts into at most one of with_pagination/with_cursor; raw_output remains mutually exclusive with both.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved public API compatibility breaks and cursor validation/rendering issues remain.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds opt-in cursor-first pagination alongside offset pagination, including cursor flags, middleware state, envelope metadata, next-page actions, human rendering, documentation, and tests.
Changes:
- Adds
CursorConfig,CursorContinuation, and--limit/--continue. - Extends envelopes and human output with cursor information.
- Adds end-to-end cursor pagination coverage and authoring guidance.
File summaries
| File | Summary |
|---|---|
cli-engine/tests/foundation.rs |
Updates middleware request fixtures. |
cli-engine/tests/cursor_pagination.rs |
Adds end-to-end cursor pagination coverage. |
cli-engine/src/output/mod.rs |
Re-exports cursor metadata. |
cli-engine/src/output/human/tests.rs |
Updates renderer tests. |
cli-engine/src/output/human/mod.rs |
Integrates cursor rendering. |
cli-engine/src/output/human/footer.rs |
Adds cursor summaries; token quoting and scalar fallback handling need changes. |
cli-engine/src/output/human/body.rs |
Adds cursor-aware table rendering. |
cli-engine/src/output/envelope.rs |
Adds CursorMeta; the public struct field is a downstream compatibility break. |
cli-engine/src/middleware/run.rs |
Builds cursor metadata and next actions; metadata should be restricted to array data. |
cli-engine/src/middleware/mod.rs |
Adds cursor state; the public field is a downstream compatibility break. |
cli-engine/src/lib.rs |
Exposes cursor APIs. |
cli-engine/src/flags/register.rs |
Registers and validates cursor arguments. |
cli-engine/src/flags/mod.rs |
Updates cursor flag handling. |
cli-engine/src/command/spec.rs |
Adds cursor configuration; the derived default currently permits an invalid zero limit. |
cli-engine/src/command/mod.rs |
Adds cursor continuation result APIs. |
cli-engine/src/cli/schema_tree.rs |
Registers cursor flags and validates combinations. |
cli-engine/src/cli/run.rs |
Applies cursor configuration during dispatch. |
cli-engine/src/cli/mod.rs |
Exposes command quoting helpers. |
cli-engine/src/cli/flags_apply.rs |
Parses and replays cursor flags. |
cli-engine/docs/concepts.md |
Documents cursor pagination; contains a spacing typo. |
AGENTS.md |
Updates cursor pagination authoring guidance. |
Review details
Suppressed comments (4)
cli-engine/src/command/spec.rs:194
CursorConfigderivesDefault, but that default hasdefault_limit == 0even though this type documents zero as invalid. In optimized builds thedebug_assert!inwith_cursoris removed, soCursorConfig::default()reaches the parser with a default--limit 0that the cursor parser rejects, leaving the command unusable. Make the default valid or enforce the invariant in release builds as well.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct CursorConfig {
/// Page size sent to the backend when the user passes no `--limit`. Must
/// be greater than zero.
pub default_limit: i64,
/// Upper bound a user can request with an explicit `--limit`. `0` (the
/// default) means uncapped. Does not affect `default_limit` itself.
pub max_limit: i64,
cli-engine/src/middleware/mod.rs:611
MiddlewareRequestis a public, non-#[non_exhaustive]struct; addingcursor_commandmakes downstream struct literals fail to compile. This needs to be handled as a breaking API change or moved behind a compatibility-preserving constructor/extension mechanism.
pub cursor_command: Option<String>,
cli-engine/src/output/envelope.rs:188
CursorMeta.limitcan be supplied byCursorContinuation::with_limit, so it is the effective page size applied by the handler, not necessarily the requested page size. The field documentation should reflect the override semantics exercised by the new API.
/// Requested page size.
pub limit: i64,
cli-engine/src/output/human/footer.rs:174
- Unlike
append_pagination_summaryimmediately above, this fallback substitutescursor.countwhenshownisNone. If--expr length(@)turns the data into a scalar, human output therefore claimsShowing 0(or a cursor "so far" count) beside non-list data; use a neutral cursor summary when no rendered array count exists, matching the pagination fallback.
let count = shown.unwrap_or(cursor.count);
out.push_str(&format!(
"\n{}\n",
cursor_summary_text(SummaryStyle::Standalone, count, cursor)
- Files reviewed: 21/21 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- CursorConfig no longer derives Default: 0 is not a valid
default_limit (unlike PaginationConfig's "0 = unlimited"), so a
release build reaching CursorConfig::default() would register an
unusable --limit whose own default value the parser then rejects,
with only a stripped debug_assert standing between the two.
- Cursor metadata (envelope.cursor + the next-page action) is now only
ever attached for array data, mirroring apply_pagination's identical
guard for offset pagination. Previously any command result --
including one --expr reshaped into a scalar/object -- got a cursor
field with a fabricated count and a next_actions entry over data
that was never actually paginated.
- append_cursor_summary's fallback for non-array shown data now prints
a neutral line instead of "Showing <stale pre-expr count> ...",
matching append_pagination_summary's existing None-branch handling.
- The human-readable "so far" summary line now quotes the resume token
the same way the auto-generated next_actions command already does;
an opaque token with shell metacharacters was otherwise unusable to
copy-paste from the sentence context.
- Fixed a docs typo ("aserver-maintained" -> "a server-maintained").
Adds 3 regression tests for the array-data guard and human-summary
quoting.
Both structs already grew fields (cursor_limit, continue_token, cursor_command) once without a compatibility escape hatch. Marking them non_exhaustive means the next such addition doesn't break external construction — but non_exhaustive forbids struct-literal syntax entirely for external callers, even with ..Default::default() spread, so this also adds a MiddlewareRequest::new constructor plus with_* builder methods, and updates foundation.rs's integration-test call sites (which compile as an external crate) to use them instead of struct literals. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved moderate findings affect streaming support, constructor usability, request invariants, and continuation instructions.
Review details
Suppressed comments (8)
Previously missed (3) — in code that hasn't changed since the last review.
cli-engine/src/cli/run.rs:519
- This forwards
cursor_commandto the streaming path, butrun_streaming_commandwraps every successful streaming handler result asCommandResult::new(Value::Null), sorender_envelopenever receives an array orCursorContinuationand can never emit cursor metadata or a next-page action. A streaming command with.with_cursortherefore advertises flags that do nothing; reject this combination or add a completion-metadata path for streaming handlers.
cli-engine/docs/concepts.md:524 - This paragraph repeats an inaccurate contract: it says
limitis always the parsed--limit, butCursorContinuation::with_limitintentionally replaces that value and causes the generated next action to omit--limit. Document this supported override here as well, otherwise consumers may assume the envelope always echoes the request.
cli-engine/src/output/envelope.rs:184 - The new
CursorMetadocumentation says bothlimitandcountare always computed from the parsed request, butCursorContinuation::with_limitintentionally overridesCursorMeta.limitand also changes next-action generation. Please document that handler-supplied effective limit here so consumers do not rely on an inaccurate contract.
cli-engine/docs/concepts.md:528
- The guide says every cursor
next_actionscommand appends both--limitand--continue, butCursorContinuation::with_limitdeliberately omits--limitwhen the token carries the effective page size. Document that exception so the guide matches the generated command.
When `continue_from` is present (`has_more`), the engine appends a `next_actions` entry replaying the command with `--limit`/`--continue <token>` for the next page.
cli-engine/src/cli/schema_tree.rs:220
- A streaming
RuntimeCommandSpeccan also carry thisCommandSpec, so this registration exposes cursor flags for a handler that returns onlyResult<()>.run_streaming_commandwraps that result asCommandResult::new(Value::Null), leaving no way to attachCursorContinuation; the stream therefore never getsenvelope.cursoror an automatic next-page action even thoughwith_cursoradvertised the feature. Reject cursor pagination for streaming constructors (or add a streaming continuation/envelope path) instead of silently accepting this unsupported combination.
command = apply_cursor_args(command, spec);
cli-engine/src/command/spec.rs:413
with_cursorcan be attached toRuntimeCommandSpec::new/new_typed, but those handlers cannot accessCommandContext, andcommand_args_from_matchesexcludes these framework-owned flags. Such a command exposes--continuebut has no way to send the parsed cursor state to its backend, so pagination cannot resume. Reject cursor specs in the non-context constructors (ashandles_dry_rundoes) or provide a handler API carrying the cursor state.
pub fn with_cursor(mut self, config: CursorConfig) -> Self {
cli-engine/src/middleware/mod.rs:681
- These public builder methods do not enforce the documented mutual-exclusion invariant: chaining
MiddlewareRequest::new(...).with_pagination_command(...).with_cursor_command(...)leaves both options set, andrender_envelopecan then run both pagination paths and emit both metadata forms/actions. Clear the opposite option in both setters, or reject the combination, so externally constructed requests cannot violate the contract.
pub fn with_cursor_command(mut self, cursor_command: impl Into<String>) -> Self {
self.cursor_command = Some(cursor_command.into());
self
cli-engine/src/output/human/footer.rs:100
- The
continue_fromsummary tells users to run only--continue <token>, but this also covers opaque tokens wherecontinuation.limitisNone; the generatednext_actionsdeliberately retains--limit {effective_limit}because resuming without it can change or invalidate the backend request. With an explicit non-default--limit, the human instruction therefore does not replay the same page and may fail. Carry the self-sufficient/limit information into the human summary (including the standalone branch) so it matches the generated next-page command.
match (style, cursor.total, cursor.remaining, &cursor.continue_from) {
(SummaryStyle::TableFooter, Some(total), _, _) => format!("{count} of {total} rows"),
(SummaryStyle::TableFooter, None, Some(remaining), _) => {
format!("{count} rows, {remaining} remaining")
}
- Files reviewed: 21/21 changed files
- Comments generated: 0 new
- Review effort level: Lite
Reject with_cursor on handler shapes that can never act on it: non-context RuntimeCommandSpec::new/new_typed (no CommandContext to read back middleware.cursor_limit/continue_token, mirroring the existing handles_dry_run guard) and streaming constructors (a streaming result is always wrapped as CommandResult::new(Value::Null), so there's no array or CursorContinuation to attach cursor metadata to). Also: MiddlewareRequest::with_pagination_command/with_cursor_command now clear each other, so external construction can't set both replayable commands at once; CursorMeta.limit and the concepts.md guide now document that a handler can override the effective page size via CursorContinuation::with_limit; and the human "so far" summary now includes --limit, matching the --limit/--continue pair the engine actually suggests for a token that isn't self-sufficient about page size. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed in 8127a6b, in response to the second Copilot review pass (
Full verification suite (fmt, clippy, tests incl. |
There was a problem hiding this comment.
🟡 Changes recommended
A critical cursor/offset state issue and a moderate next-page hint inconsistency remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 22/22 changed files
- Comments generated: 2
- Review effort level: Lite
…ar hint apply_cursor_flags left middleware.limit/offset untouched, so a value set by a prior with_pagination command on the same long-lived Middleware (Cli owns one across repeated run() calls, and it's mutable via Cli::middleware_mut) would survive into a cursor command's run. apply_pipeline slices on limit > 0 || offset > 0 with no idea which pagination style the current command declared, so a stale value would silently client-slice a response the handler already computed the exact requested page for. Cursor commands now zero both fields before running. Also add CursorMeta.self_sufficient_limit, set from whether the handler called CursorContinuation::with_limit, so the human "so far" resume hint can match next_actions exactly: include --limit only when the token isn't self-sufficient about page size, omit it when it is. The previous fix (always show --limit) was itself wrong for the self-sufficient case, where a handler-reported effective limit can exceed the command's own max_limit and get rejected by the parser if replayed literally. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved moderate issues affect cursor replay safety, field preservation, public API compatibility, and pipeline behavior.
Review details
Suppressed comments (4)
cli-engine/src/cli/flags_apply.rs:216
- The cursor token is now sourced from a backend and passed through this helper for both
next_actionsand human output, but the quoted branch leaves control characters such as ESC, CR, and LF unchanged. A backend-supplied token containing an ANSI sequence can alter the terminal, and embedded newlines make the copy-paste command ambiguous even though shell metacharacters are escaped. Escape non-printable/control characters before rendering the token and add a regression test for this path.
pub(crate) fn quote_pagination_value(value: &str) -> String {
let safe_unquoted =
|c: char| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/' | ':' | '@');
if value.is_empty() || !value.chars().all(safe_unquoted) {
let escaped = value
.replace('\\', "\\\\")
.replace('"', "\\\"")
.replace('$', "\\$")
.replace('`', "\\`");
format!("\"{escaped}\"")
cli-engine/src/cli/flags_apply.rs:113
command_replay_basenow also feeds cursornext_actions, but it ignoresflags.fields_explicit. For a command withdefault_fields,flags.fieldsis non-empty even when the user did not pass--fields, so the suggested cursor command turns an author-controlled default into an explicit projection; the first run skips validation for that default, while the replay can fail when an optional field is absent. An explicit--fields ""is also dropped and reverts to the default on the next page. Only replay an explicit fields value, preserving an explicit empty value.
pub(super) fn command_replay_base(
binary_name: &str,
command_path: &str,
cli-engine/src/command/spec.rs:197
CursorConfigis a new public configuration struct, but it is exhaustive and the documentation encourages consumers to construct it with a literal. Any future configuration field will therefore break every consumer, unlike the non-exhaustive public authoring types introduced elsewhere in this PR. Please make this type non-exhaustive and provide a constructor/builder (or another future-compatible construction pattern) before exposing it.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct CursorConfig {
/// Page size sent to the backend when the user passes no `--limit`. Must
/// be greater than zero.
pub default_limit: i64,
/// Upper bound a user can request with an explicit `--limit`. `0` (the
/// default) means uncapped. Does not affect `default_limit` itself.
pub max_limit: i64,
cli-engine/src/middleware/run.rs:574
- Although
apply_cursor_flagsclearsmiddleware.limit/offset,run_pre_runexecutes after that reset and can legally mutate the public middleware state; callers ofMiddleware::runcan also preset those fields.apply_pipelinewill then slice the backend page before this cursor block, potentially dropping rows and even producing bothpaginationandcursormetadata. Make the pipeline use zero limit/offset whenevercursor_command.is_some()(rather than relying only on the earlier reset).
if let Some(base) = cursor_command
&& let Some(data) = &envelope.data
&& let Some(items) = data.as_array()
- Files reviewed: 22/22 changed files
- Comments generated: 0 new
- Review effort level: Lite
…nation quote_pagination_value only escaped shell metacharacters — a backend- supplied cursor token containing a raw newline or ANSI escape sequence could still make the printed next-page suggestion look like multiple lines or repaint the terminal when displayed. Control characters now render as \n/\r/\t or a \xHH hex placeholder instead of passing through raw. Also make the pipeline itself, not just apply_cursor_flags, the authoritative place a cursor command's response is never client-sliced: a pre_run hook (a legitimate extension point) runs after apply_cursor_flags and could still set middleware.limit/offset, and a caller driving Middleware::run directly bypasses apply_cursor_flags entirely. Forcing limit/offset to zero whenever cursor_command.is_some(), right where PipelineOpts is built, closes that regardless of how the stale state got there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed 2 of 4 findings from the fourth Copilot review pass (
The remaining 2 findings ( Full verification suite passes on 7b75d7a. |
Both are plain public structs documented and used via literal construction, so a future field addition would break every external caller — the same problem this PR already hardened Middleware/MiddlewareRequest against. CursorConfig was the specific Copilot finding; PaginationConfig is fixed alongside it for API consistency between the two sibling pagination configs, rather than leaving one non_exhaustive and the other not. Add a plain new(default_limit, max_limit) constructor to each (no with_* builders — both fields are always required, nothing to build up incrementally) and convert every literal-construction call site (tests, AGENTS.md, docs) to use it, since non_exhaustive forbids struct-literal syntax entirely for external callers, spread syntax included. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 2 remaining findings from the fourth Copilot review pass (
Full verification suite passes on d68b1e2. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved API compatibility, opaque-token parsing/replay, and custom-renderer output issues block approval.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (5)
cli-engine/src/cli/flags_apply.rs:227
quote_pagination_valueis also used bycommand_replay_basefor ordinary user-argument replay, not only for cursor tokens. Replacing a control byte with the text\\n/\\xHHmeans the shell passes those literal characters (the CLI does not decode them), so a generated next-page command cannot round-trip an argument or opaque token containing a control character. Either emit a representation that reconstructs the original value or suppress the replay action when exact reproduction is impossible.
'\n' => escaped.push_str("\\n"),
'\r' => escaped.push_str("\\r"),
'\t' => escaped.push_str("\\t"),
c if c.is_control() => escaped.push_str(&format!("\\x{:02x}", c as u32)),
cli-engine/src/flags/register.rs:307
- Because this accepts arbitrary hyphen-prefixed tokens,
--continue --schemais a valid cursor invocation, butCli::runcalls the rawhas_true_schema_flagscan before clap parses the command and mistakes that token for the global schema flag. The handler is then skipped and schema output is returned instead of fetching the requested cursor page. The raw pre-scan must skip the value following--continue(and other per-command value flags), or schema detection must be deferred until parsed matches; add a regression for a token such as--schema.
Arg::new("continue")
.long("continue")
.value_name("TOKEN")
.allow_hyphen_values(true)
cli-engine/src/flags/register.rs:307
- Cursor tokens are documented as opaque strings, but the command-line normalization pass runs before clap and does not know that
--continueconsumes the next token. For example,my-cli list --continue --verbose xis rewritten as if--verbosewere a global flag (and--continue --schemabecomes--schema=true), so the token is changed or the invocation fails before the handler sees it. The pre-parser must consume cursor values using the command tree's value-argument rules before applying optional-global normalization.
Arg::new("continue")
.long("continue")
.value_name("TOKEN")
.allow_hyphen_values(true)
cli-engine/src/flags/register.rs:307
- When environments are configured,
Cli::newalso runsprescan_env_flagover the raw argv before the command is parsed. A valid opaque token such as--continue --env devis therefore treated as the invocation's global environment override, which can select/prune a different command tree before dispatch. That prescan must likewise know that--continueconsumes its following token (or cursor tokens must be constrained/encoded).
Arg::new("continue")
.long("continue")
.value_name("TOKEN")
.allow_hyphen_values(true)
cli-engine/src/output/human/mod.rs:447
- The new cursor footer is only reached through
render_human_with_view, butrender_human_with_registry_selectedreturns early for a registered custom renderer after appending onlynext_actions. A valid cursor command usingregister_functherefore gets no documented standalone cursor summary in human output, even though custom output is a non-table response. Apply the cursor summary footer in that custom-renderer path as well (using the array length) before appending next actions.
append_cursor_summary(&mut body, envelope.cursor.as_ref(), shown);
- Files reviewed: 25/25 changed files
- Comments generated: 1
- Review effort level: Lite
…tradeoff Addresses a Copilot review finding: escaping a control character to a literal \n/\xHH placeholder (added for terminal display safety) means a shell won't decode it back, so a replayed value containing one can't be copy-pasted into an exact resend. Documenting this as a deliberate trade-off rather than fixing it — the alternative (ANSI-C $'...' quoting) isn't POSIX and would make every other, ordinary replayed value non-portable to gain exact reproduction for a case only a malformed or adversarial backend token would hit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 3 findings from the fifth Copilot review pass (
Full verification suite passes on e876802. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved findings cover TOON control-character escaping, Bash-safe ! replay quoting, explicit empty --fields replay, PaginationConfig compatibility, and stale cursor envelope documentation.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (4)
cli-engine/docs/proposals/cursor-first-pagination.md:87
- This example update leaves the proposal's envelope section stale: it still describes cursor metadata as making
offsetoptional, while the implemented publicCursorMetahas nooffsetand instead addsremainingandself_sufficient_limit. Please update that section as part of this proposal change so it documents the actual API consumers receive.
CommandSpec::new("list", "List things").with_cursor(CursorConfig::new(25, 500))
cli-engine/src/cli/flags_apply.rs:232
- The quoted branch still leaves
!unescaped inside a double-quoted value. In interactive Bash, history expansion is performed inside double quotes, so a valid backend token such asa!bcan expand to a history entry or fail withevent not found; the suggested next-page command is then not safely copy-pastable or an exact replay. Use a quoting form that also suppresses Bash history expansion (for example POSIX single-quote escaping for values containing!).
'\\' => escaped.push_str("\\\\"),
'"' => escaped.push_str("\\\""),
'$' => escaped.push_str("\\$"),
'`' => escaped.push_str("\\`"),
cli-engine/src/cli/flags_apply.rs:111
command_replay_basecannot distinguish an explicit--fields ""from no--fields: it only emits non-emptyflags.fieldsand does not consultflags.fields_explicit. For a command withdefault_fields, the current invocation keeps all fields while the suggested next-page command falls back to the command default, so the cursor replay is not equivalent. Preserve an explicit empty--fieldsin the replay.
pub(super) fn command_replay_base(
cli-engine/src/command/spec.rs:156
- Marking the existing public
PaginationConfigas#[non_exhaustive]is a source-breaking change: all of its fields were public and the previous docs explicitly supportedPaginationConfig { ..., ..Default::default() }, but downstream struct literals can no longer compile. This conflicts with the proposal's additive/non-breaking framework claim; keep this pre-existing type constructible (and reserve#[non_exhaustive]for the new cursor type), or explicitly version and document the breaking change.
#[non_exhaustive]
- Files reviewed: 25/25 changed files
- Comments generated: 1
- Review effort level: Lite
|
Resolving the 4 findings from the sixth Copilot review pass (
Full verification suite passes on e7ee3c5. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved API compatibility, TOON control-character escaping, and custom-renderer summary issues remain.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (3)
cli-engine/src/command/spec.rs:156
- Adding
#[non_exhaustive]to the existing publicPaginationConfigis source-breaking: consumers following the previous documentedPaginationConfig { ... ..Default::default() }pattern can no longer construct it, even though the proposal still describes cli-engine as additive/non-breaking and the summary says offset pagination is untouched. Either preserve the old construction path, or explicitly treat this as a semver/migration change and update the proposal/release docs.
#[non_exhaustive]
cli-engine/src/output/envelope.rs:204
continue_fromis backend-controlled and is still serialized into every output format, but the TOON renderer'sis_safe_unquoted/escape_stringpath does not reject or escape control characters such as ESC. Thus--output tooncan print an ANSI sequence from this new field directly to a terminal, even though the suggested command is escaped; make the TOON string encoding reject/escape all control characters (or otherwise sanitize this field) before emitting it.
pub continue_from: Option<String>,
cli-engine/src/output/human/mod.rs:447
- The new cursor summary is only appended after
render_data_body, butrender_human_with_registry_selectedreturns early for a custom human renderer before reaching this path. A cursor command usingregister_global_human_view_func/HumanViewRegistry::register_functherefore shows no cursor count, limit, or resume hint (including on the last page), even though the feature's human-output contract says these facts are rendered; append the cursor summary in the custom-renderer branch as well, using the final array count, before its existing next-actions footer.
append_cursor_summary(&mut body, envelope.cursor.as_ref(), shown);
- Files reviewed: 25/25 changed files
- Comments generated: 1
- Review effort level: Lite
is_safe_unquoted only forced the quoted path for \n/\r/\t, and escape_string only escaped those three plus \\/" — any other control character (e.g. ESC, the start of most ANSI escape sequences) rendered completely unquoted and unescaped, straight to a terminal running --output toon. Not specific to cursor pagination's continue_from (any string field goes through this same encoder), but this PR's opaque, backend-controlled token is what surfaced it: unlike the shell-quoting fix for the suggested next-page command, nothing sanitized the raw field value itself in TOON's own encoding. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 3 findings from the seventh Copilot review pass (
Full verification suite passes on b529d41. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical runtime validation and moderate compatibility, metadata-count, and TOON-escaping findings block approval.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (6)
cli-engine/docs/design.md:296
- This overview still describes
--limit/--offsetas the only per-command pagination exception, but this PR adds the parallelwith_cursor/--continueAPI. The output-envelope and pipeline summaries later indesign.mdlikewise omitcursor, so readers of this public design document get an incomplete contract. Please update this section and the output summary to describe both mutually exclusive styles.
for itself with `CommandSpec::with_pagination(PaginationConfig::new(default_limit, max_limit))`;
cli-engine/src/command/runtime.rs:137
- In release builds this
debug_assert!is removed, so a streaming command can still advertise--limit/--continue; however, the streaming path always returnsValue::Nulland has noCursorContinuationto drive a next page. That makes the advertised cursor flags silently ineffective. Reject this pairing unconditionally (includingnew_typed_streaming).
debug_assert!(
spec.cursor.is_none(),
"command {:?} sets with_cursor but a streaming handler's result is always wrapped \
as CommandResult::new(Value::Null) — there is no array or CursorContinuation to \
report, so --continue would advertise resumption that can never happen; \
with_cursor is only supported on non-streaming commands",
spec.name
);
cli-engine/src/command/spec.rs:156
- Marking the existing
PaginationConfigas#[non_exhaustive]is a source-breaking change: consumers following the previously documentedPaginationConfig { default_limit, max_limit, ..Default::default() }pattern can no longer compile, even though this PR describes the offset pagination API as untouched/additive. Keep this existing public struct constructible or explicitly version/document the breaking migration rather than treating the new constructor as compatibility-preserving.
#[non_exhaustive]
cli-engine/src/command/spec.rs:124
- This field's
Nonedescription is now misleading: a cursor command haspagination == Nonebut still registers the shared--limit. The siblingcursordescription has the analogous ambiguity for offset commands. Please document these as independent offset/cursor policies (for example,pagination == Nonemeans no--offset, not no pagination), otherwise generated rustdoc gives incorrect guidance for the new API.
/// [`with_pagination`](CommandSpec::with_pagination). Mutually exclusive
/// with [`cursor`](CommandSpec::cursor).
cli-engine/src/middleware/run.rs:601
- This count is taken after
apply_pipeline, so an expression that keeps an array (for example--expr "[?name=='alpha']") changescursor.countfrom the backend page size to the post-expression display size. The cursor metadata is meant to describe the handler's returned page (and the footer comments already distinguish that from the displayed count); capture the count before--exprand use the post-pipeline length only for human rendering.
let count = items.len() as i64;
let continuation = cursor_continuation.unwrap_or_default();
let has_more = continuation.continue_from.is_some();
cli-engine/src/output/toon.rs:324
- The new
\xHHescape is not a JSON-style string escape, while this renderer's quoted strings otherwise use JSON-style escapes (\\,\",\n, etc.). A cursor token containing ESC therefore renders as invalid TOON such as"...\\x1b...", defeating machine-readable TOON output. Encode arbitrary control characters with a TOON-supported Unicode escape (for example\\u{:04x}) instead.
c if c.is_control() => escaped.push_str(&format!("\\x{:02x}", c as u32)),
- Files reviewed: 26/26 changed files
- Comments generated: 1
- Review effort level: Lite
…gaps cursor.count was measured from envelope.data after apply_pipeline ran, so an --expr that filters but keeps the result an array (e.g. a JMESPath predicate) silently substituted the post-expression display count for the page the handler's backend call actually returned. PaginationMeta.count already avoids this — apply_pipeline captures it internally, before --expr runs — but cursor metadata is built entirely outside apply_pipeline, so it needed its own pre-pipeline snapshot. Added a regression test filtering a 2-item page down to 1 displayed row and asserting count still reads 2. Also fixed \xHH from the previous TOON control-character fix: not a valid JSON/TOON string escape (every other escape there is JSON-style), so it would itself make the rendered output unparseable. Uses \uXXXX now, matching the rest of the encoder. Doc fixes: CommandSpec::pagination/cursor field docs corrected (each being None doesn't mean no --limit at all — the other might still register one), and docs/design.md now describes with_cursor alongside with_pagination instead of omitting it entirely. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 6 findings from the eighth Copilot review pass (
Full verification suite passes on 8ec8c7e. |
There was a problem hiding this comment.
🟡 Changes recommended
Release-only cursor guards, an API compatibility break, missing #[must_use] annotations, and cursor validation issues remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (6)
Previously missed (1) — in code that hasn't changed since the last review.
cli-engine/src/middleware/run.rs:607
- This guard only checks the post-pipeline value. If a handler returns an object or scalar with
with_cursorand--exprtransforms it into an array,raw_cursor_array_lenisNonebut this block still attachesCursorMetaand can emit a next-page action for data that was never a cursor page. Require the pre-pipeline array snapshot to be present before entering this block.
cli-engine/src/command/runtime.rs:74
debug_assert!is removed from release builds, so a production consumer can still construct a cursor command withRuntimeCommandSpec::new. This handler only receives the resolver andspec.args; the cursor flags are registered separately and are not in that map, so it cannot observe--continueand the generated next-page action cannot actually resume. Make this rejection unconditional (or reject the spec during registration).
debug_assert!(
cli-engine/src/command/runtime.rs:180
- This has the same release-build problem as the untyped constructor:
debug_assert!disappears, allowingRuntimeCommandSpec::new_typedto register cursor flags even though its handler receives noCommandContextand cannot read the parsed continuation token. A generated next-page command will not be resumable; enforce the rejection in all builds.
debug_assert!(
cli-engine/src/command/runtime.rs:282
debug_assert!disappears in release builds here as well, allowing a typed streaming cursor command whose wrapped result is alwaysNulland can never report a continuation. In that build the command accepts--continuebut silently ignores the cursor contract; enforce this constructor guard unconditionally or reject it at registration.
debug_assert!(
cli-engine/src/command/spec.rs:164
PaginationConfigwas previously an exhaustive public struct with public fields, and its documented construction pattern used a struct literal with..Default::default(). Adding#[non_exhaustive]makes existing downstream offset-pagination code fail to compile, so this is a breaking change to the existing API even though the new cursor type is the only type that needs future-proofing. KeepPaginationConfigexhaustive (and retain the old literal-compatible documentation), or version and explicitly announce this migration.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
#[non_exhaustive]
cli-engine/src/middleware/mod.rs:635
- The new consuming constructor/builders omit
#[must_use], unlike the existing public constructors/builders (for exampleMiddleware::newinsrc/middleware/run.rs:21-24andCommandSpec::with_longinsrc/command/spec.rs:245-249). IgnoringMiddlewareRequest::new(...).with_auth(...)or anotherwith_*call silently discards the configured request; add#[must_use]to the constructor and all of these fluent methods.
impl<'request> MiddlewareRequest<'request> {
/// Builds a request from the fields every caller needs to set, with
/// everything else defaulted (`auth: AuthRequirement::Required`,
/// `view_id`/`pagination_command`/`cursor_command`: `None`, `raw_output:
/// false`). Chain the `with_*` methods below for anything else.
pub fn new(
- Files reviewed: 26/26 changed files
- Comments generated: 1
- Review effort level: Lite
…ust_use] The round-8 fix for cursor.count's --expr timing bug introduced its own gap: the guard still only checked the post-pipeline shape, so a handler that returned a non-array result (never a real cursor page) could still get cursor metadata and a next-page action attached if --expr happened to synthesize an array from it (e.g. [@], wrapping a scalar/object in a single-element list). Both checks are now required: the pre-pipeline snapshot must be Some (the handler's raw result was an array) AND the post-pipeline data must still be an array (catches the opposite direction, --expr reshaping a real page into a scalar). Added a regression test for the newly-caught direction. Also added #[must_use] to MiddlewareRequest::new and its with_* builders, matching the existing convention on this crate's other consuming constructors/builders (Middleware::new, PaginationConfig::new, ...). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 6 findings from the ninth Copilot review pass (
Full verification suite passes on 3f9898c. |
There was a problem hiding this comment.
🔵 Needs a closer look
Resolve the PaginationConfig compatibility break and prevent completed continuations from being marked self-sufficient without a token.
Review details
Suppressed comments (2)
cli-engine/src/command/spec.rs:164
- Marking the existing public
PaginationConfigas#[non_exhaustive]is a source-breaking change: downstream callers that used the previously documentedPaginationConfig { default_limit: ..., ..Default::default() }form can no longer compile, even though this PR describes offset pagination andPaginationConfigas untouched/fully supported. A constructor does not preserve that API. Please either leave this existing type exhaustive (only the newCursorConfigneeds future-proofing) or explicitly treat this as a breaking release/migration.
#[non_exhaustive]
cli-engine/src/middleware/run.rs:655
self_sufficient_limitis documented as describing whethercontinue_fromalone can resume, but this is set fromcontinuation.limiteven when the continuation isdone()andcontinue_fromisNone(for example,CursorContinuation::done().with_limit(2)). That produces cursor metadata claiming a nonexistent token is self-sufficient. Set this flag only when a continuation token is present, or reject/ignorewith_limitfor a completed page.
self_sufficient_limit: continuation.limit.is_some(),
- Files reviewed: 26/26 changed files
- Comments generated: 0 new
- Review effort level: Lite
CursorContinuation::done().with_limit(n) is a handler misuse — there's no continue_from for n to describe as self-sufficient — but self_sufficient_limit was set purely from continuation.limit.is_some(), so this would claim a nonexistent token is self-sufficient about page size. Gated it on has_more (already computed, and continuation.continue_from itself is unavailable by this point in the struct literal since it's moved into the continue_from field first). Added a regression test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 2 findings from the tenth Copilot review pass (
Full verification suite passes on 08018f2. |
There was a problem hiding this comment.
🔵 Needs a closer look
Address the PaginationConfig compatibility issue and the two documentation nits.
Review details
Suppressed comments (3)
cli-engine/docs/concepts.md:517
- This paragraph says
self_sufficient_limitistrueexactly when the handler calledwith_limit, but the implementation deliberately sets it false forCursorContinuation::done().with_limit(...)because nocontinue_fromtoken exists. Please document the actualhas_more && with_limitcondition, which the new completed-page test asserts.
A command that opted into `--limit`/`--continue` cursor pagination via `CommandSpec::with_cursor` gets a top-level `cursor` field on the envelope instead of `pagination` — `limit`, `count`, `total`, `remaining`, `continue_from`, `has_more`, and `self_sufficient_limit` — whenever it returned array data. Unlike `pagination`, the engine cannot compute this itself: a cursor is opaque to everything except the handler that called the backend, so `count` is the only piece the engine derives itself (the returned array's length); `limit` defaults to the parsed `--limit`, but a handler can override it via `CursorContinuation::with_limit` to report the effective page size it actually resumed with (e.g. one decoded from `continue_from` itself) — `self_sufficient_limit` is `true` exactly when that override happened, meaning `continue_from` alone is enough to resume and a replay command can omit `--limit`; `total`/`remaining`/`continue_from` come from whatever the handler reported via `CommandResult::with_cursor(CursorContinuation::more(token).with_total(n).with_remaining(n))` — or `CursorContinuation::done()` (or no call at all) to report the end of iteration. `total`/`remaining` are `None` when the backend never reports them, which a pure opaque-cursor API is not obligated to do.
cli-engine/src/command/spec.rs:164
- Marking the existing
PaginationConfigas#[non_exhaustive]is source-breaking for downstream commands that use the previously documentedPaginationConfig { default_limit, max_limit, ..Default::default() }construction; addingnewdoes not preserve that API. This also contradicts the proposal's additive/non-breaking contract forwith_pagination. Please keep the existing config constructible or make this breaking change explicit in the release/versioning plan.
#[non_exhaustive]
cli-engine/src/output/envelope.rs:212
- This says
self_sufficient_limitis true when the handler calledwith_limit, but the implementation deliberately sets it false forCursorContinuation::done().with_limit(...)because nocontinue_fromtoken exists. Please document the actualhas_more && with_limitcondition, which the new completed-page test asserts.
/// Whether `continue_from` alone is sufficient to resume at `limit`
/// (the handler called
/// [`CursorContinuation::with_limit`](crate::CursorContinuation::with_limit)),
/// so a replay command can omit `--limit` — the same condition the
/// engine uses to decide whether `next_actions` includes it. `false` for
/// a plain [`CursorContinuation::more`](crate::CursorContinuation::more)
- Files reviewed: 26/26 changed files
- Comments generated: 0 new
- Review effort level: Lite
Fallout of the previous fix (self_sufficient_limit is now has_more && continuation.limit.is_some(), not just the latter) — both the CursorMeta field doc and concepts.md's cursor-pagination section still described the old, incomplete condition. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Resolving the 3 findings from the eleventh Copilot review pass (
Full verification suite passes on f33aa2a. |
There was a problem hiding this comment.
🔵 Needs a closer look
Unresolved moderate findings remain in replay-value quoting, release-build guards, and PaginationConfig API compatibility.
Review details
Suppressed comments (6)
cli-engine/src/cli/flags_apply.rs:254
- These control-character escapes are applied by the shared helper to every replayed value, not just backend cursor tokens.
command_replay_baseuses it for user-supplied command args and--filter/--expr/--fields; a value containing a newline, carriage return, or tab was previously preserved inside the double-quoted shell word, but now becomes the literal characters\\n/\\r/\\t, so the suggested next-page command no longer replays the request. Keep control sanitization in a cursor-token-specific quoting path (or parameterize this helper) while preserving round-trip quoting for user arguments.
'\n' => segment.push_str("\\n"),
'\r' => segment.push_str("\\r"),
'\t' => segment.push_str("\\t"),
c if c.is_control() => segment.push_str(&format!("\\x{:02x}", c as u32)),
cli-engine/src/command/runtime.rs:83
- This guard disappears in optimized builds, so a release consumer can register
with_cursorwithRuntimeCommandSpec::new. That handler only receivesspec.args(the generated cursor flags are not in that list), so it cannot observecursor_limitorcontinue_tokenand will generally ignore--continue, repeatedly fetching the initial page despite the advertised cursor flags. The PR contract says this pair is rejected at construction time; make the rejection unconditional or otherwise prevent this invalid pairing in release builds.
debug_assert!(
spec.cursor.is_none(),
"command {:?} sets with_cursor but RuntimeCommandSpec::new's handler \
(CredentialResolver, args) has no CommandContext and can never read back \
middleware.cursor_limit/continue_token to drive its own backend call, so \
--continue would advertise resumption the handler cannot perform; use \
RuntimeCommandSpec::new_with_context (or new_typed_with_context to keep typed \
args) instead",
spec.name
);
cli-engine/src/command/runtime.rs:189
- The same release-build hole exists for
new_typed: afterdebug_assert!is removed, a cursor command can be constructed with a handler that receives only the typed args and has no middleware context. It therefore cannot consume--continue, while the command still registers and advertises cursor pagination. This guard needs to be enforced in release builds or the invalid constructor pairing must be made impossible.
debug_assert!(
spec.cursor.is_none(),
"command {:?} sets with_cursor but RuntimeCommandSpec::new_typed's handler \
(CredentialResolver, args) has no CommandContext and can never read back \
middleware.cursor_limit/continue_token to drive its own backend call, so \
--continue would advertise resumption the handler cannot perform; use \
RuntimeCommandSpec::new_with_context (or new_typed_with_context to keep typed \
args) instead",
spec.name
);
cli-engine/src/command/runtime.rs:137
- This streaming guard is also debug-only. In an optimized build, a streaming command can opt into cursor flags, but the streaming adapter always returns
CommandResult::new(Value::Null), so no cursor envelope or next-page action can ever be produced. That leaves--continueadvertised without the promised cursor behavior; enforce the restriction outsidedebug_assert!or remove the unsupported pairing from the public API.
debug_assert!(
spec.cursor.is_none(),
"command {:?} sets with_cursor but a streaming handler's result is always wrapped \
as CommandResult::new(Value::Null) — there is no array or CursorContinuation to \
report, so --continue would advertise resumption that can never happen; \
with_cursor is only supported on non-streaming commands",
spec.name
);
cli-engine/src/command/runtime.rs:289
- This typed streaming guard is debug-only as well. In an optimized build, the command still registers
--continue, but the streaming adapter wraps the handler's completion asCommandResult::new(Value::Null), so the cursor metadata and automatic next-page action can never be emitted. Enforce this unsupported pairing in release builds or make it impossible through the constructor API.
debug_assert!(
spec.cursor.is_none(),
"command {:?} sets with_cursor but a streaming handler's result is always wrapped \
as CommandResult::new(Value::Null) — there is no array or CursorContinuation to \
report, so --continue would advertise resumption that can never happen; \
with_cursor is only supported on non-streaming commands",
spec.name
);
cli-engine/src/command/spec.rs:164
- Adding
#[non_exhaustive]to the existingPaginationConfigis an unnecessary source-breaking change: downstream callers that currently usePaginationConfig { ... }(including..Default::default()) will stop compiling even though this type gained no field in this PR. Keep the existing config constructible (only the newCursorConfigneeds future-field protection), or explicitly treat this release as a breaking API version.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
#[non_exhaustive]
- Files reviewed: 26/26 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
The twelfth Copilot review pass (
No code change accompanies this reply — per this loop's own convention (only re-request a Copilot review when a round actually changes code), this is the natural stopping point. All genuinely new, actionable findings across the previous 11 rounds have been fixed and verified; what remains are explicitly-decided trade-offs and an established crate-wide convention this PR is consistent with, not open bugs. |
Summary
docs/proposals/cursor-first-pagination.md:CommandSpec::with_cursor/CursorConfigregister--limit/--continueas a new, parallel pagination mechanism.CommandSpec::with_pagination/PaginationConfig(--limit/--offset) is untouched and fully supported — a command opts into at most one of the two.CommandResult::with_cursor(CursorContinuation), which surfaces as a newenvelope.cursorfield (CursorMeta) and an automaticnext_actions"next page" suggestion, mirroring the offset-pagination machinery end to end (flag registration, middleware state, envelope construction, human-output rendering, docs).CursorContinuation::with_limitlets a handler report an effective page size that differs from the parsed--limit(e.g. derived from the--continuetoken itself, so a caller can resume with--continuealone without repeating--limit) — this also signals the engine to omit--limitfrom the auto-generated next-page command, since the token is then self-sufficient about size.raw_outputremains mutually exclusive with both pagination styles.MiddlewareandMiddlewareRequestare now#[non_exhaustive](addressed a Copilot review finding): both structs already grew a field once without any compatibility escape hatch for external callers.MiddlewareRequestgains anew()constructor pluswith_auth/with_view_id/with_raw_output/with_pagination_command/with_cursor_commandbuilders, since#[non_exhaustive]forbids struct-literal construction (even..Default::default()spread) from outside this crate.PaginationConfig/CursorConfigare also now#[non_exhaustive]with a plainnew(default_limit, max_limit)constructor each, for the same reason and for API consistency between the two sibling pagination configs.RuntimeCommandSpec::new/new_typed) or streaming constructor — neither can act on--continue— mirroring the existinghandles_dry_runmisuse guard. A cursor command's response is also never client-sliced by stale offset-pagination state, even if apre_runhook or a directMiddleware::runcaller setslimit/offsetafter flag parsing. A backend-supplied--continuetoken with control characters (e.g. an ANSI escape sequence) is escaped before it's ever printed in a suggested next-page command or a raw--output toonfield, and a bare!is spliced into its own single-quoted segment so a copy-pasted suggestion can't trigger interactive Bash history expansion.envelope.cursor.countreflects the handler's actual returned page size, not a stale/--expr-reshaped display count — captured before the output pipeline runs, mirroring how offset pagination already capturesPaginationMeta.count.CursorMeta.self_sufficient_limitrecords whethercontinue_fromalone (viaCursorContinuation::with_limit) is enough to resume atlimit, so the human "so far" hint always matches what the generated next-page command actually needs.Test plan
cargo fmt --all --checkcargo clippy --all-targets -- -D warningsRUSTDOCFLAGS='-D warnings' cargo doc --no-depscargo test --all-targets(newtests/cursor_pagination.rs, 26 end-to-end cases, plus updates to existing pagination/human-output/TOON tests)cargo rustdoc --lib -- -W missing-docs(zero missing docs)./cli-engine/scripts/check-module-size.shManual verification
Exercised end-to-end against
gddywith itscli-enginedependency patched to this branch. Migrated all 8 of its.with_paginationcommands to.with_cursor, covering every backend shape this feature needs to support:domain list's v3pageToken)email list,dns list)platform app list'sfirst/after)platform actions list,api search/response list/parameter list)Confirmed via
--helpand live runs that:--continueshows up only for commands that opted in;--limit/--continueare rejected as unknown args otherwise; a suggested next-page command round-trips correctly across multiple pages; and--limitis correctly omitted from the suggestion exactly when (and only when) a handler's token is self-sufficient about page size.