Skip to the content.

CLike Harper Telemetry Portal

Overview

The CLike Harper Telemetry Portal is a lightweight operational dashboard for inspecting Harper pipeline execution data across IDEA, SPEC, PLAN, KIT, EVAL, GATE, and FINALIZE phases.

It provides engineering teams with a centralized view of model usage, run duration, token consumption, generated artifacts, evaluation outcomes, and gate decisions. The portal supports the CLike objective of making AI-native development observable, auditable, and continuously improvable.

Default local URL:

http://localhost:8000/v1/metrics/harper/ui

Access. The page itself is public on loopback, but the telemetry API requires the CLike service token. On first load the portal asks for CLIKE_API_TOKEN and exchanges it for an HttpOnly, SameSite=Strict cookie scoped to /v1/metrics (8 hours). POST /v1/metrics/logout clears it.

The portal is served by the Gateway telemetry routes under:

/v1/metrics/harper

Functional Purpose

The Telemetry Portal helps product engineers and platform owners answer practical questions such as:

The portal is not only a reporting UI. It is part of the Eval-Driven Development feedback loop: telemetry data helps identify bottlenecks, model quality issues, prompt regressions, and pipeline defects.


Local-agent runs

Harper phases run by Claude Code or Codex do not call a provider through the gateway: the orchestrator reports them with POST /v1/harper/telemetry (from /local-agent/complete). Their rows have execution: "local_agent", provider/executor = claude_code | gpt_codex, the model reported by the agent, usage, duration_ms and pricing.total_cost as the API-equivalent cost (unit: usd_api_equivalent; subscription runs are not billed per token). Claude Code reports that cost itself. Codex does not: the portal estimates it from the model’s price in configs/models.yaml (input_per_1k, cached_input_per_1k, output_per_1k; Codex counts cache reads inside input_tokens), marked pricing.estimated: true, also for rows written before. A model without a price (or a wrong model name) stays at zero.

Prompt cache. Costs include the provider prompt cache, priced from configs/models.yaml (OpenAI and Anthropic list prices): OpenAI counts cached input inside input_tokens (input_tokens_details.cached_tokens) and it is charged at cached_input_per_1k; Anthropic reports cache reads (cache_read_input_tokens, cached_input_per_1k) and writes (cache_creation_input_tokens, cache_write_per_1k for the 5-minute TTL, cache_write_1h_per_1k for 1 hour) apart from input_tokens. Each row’s pricing has cached_tokens, cached_cost, cache_write_tokens, cache_write_cost; the aggregate has tokens_cached per phase and day and total_tokens_cached, and the raw table a Cached column. Rows recorded before this (OpenAI cache charged as normal input, Anthropic cache not charged) are recomputed when shown (pricing.cache_recomputed).

Requirements, drill-down, KPIs and estimate

Four views over the selected project or file:

Endpoint: GET /v1/metrics/harper/reqs?project_id=…&estimate_reqs=N (reqs_file?relpath=… for one file). Rows of an eval or gate (execution: eval_sandbox) count as eval time; the others as agent/model time.

Technical Architecture

The telemetry UI is exposed by the Gateway service.

VS Code Extension
  -> Orchestrator
      -> Gateway
          -> /v1/metrics/harper/ui
          -> /v1/metrics/harper/*

Telemetry files are read from the configured Harper telemetry directory:

HARPER_TELEMETRY_DIR

Default directory:

/workspace/telemetry

Each Harper phase can emit telemetry containing:


Main Endpoints

Telemetry UI

GET /v1/metrics/harper/ui

Browser-based dashboard for interactive telemetry inspection.

Projects

GET /v1/metrics/harper/projects

Lists projects with available telemetry data.

Files

GET /v1/metrics/harper/files

Lists raw telemetry files available to the Gateway.

Aggregate Metrics

GET /v1/metrics/harper/aggregate?project_id=<PROJECT_ID>

Returns aggregated metrics for a project.

Typical use cases:

Time Series

GET /v1/metrics/harper/series?project_id=<PROJECT_ID>

Returns time-series telemetry for trend analysis.

Typical use cases:

Top Runs

GET /v1/metrics/harper/top?project_id=<PROJECT_ID>&limit=10

Returns top runs by cost, duration, token usage, or other ranking criteria exposed by the Gateway.

Raw Telemetry

GET /v1/metrics/harper/raw?project_id=<PROJECT_ID>

Returns raw telemetry records for debugging or external analysis.


Why It Matters

CLike is designed as an AI-native software engineering pipeline where human intent is translated into verifiable software through short, governed iterations.

Telemetry is essential because AI-generated software must not be treated as a black box. Every run should be observable and explainable.

The portal supports:


Example Workflow

A typical usage flow is:

/spec
/plan
/kit REQ-003
/eval REQ-003
/gate REQ-003

Then open:

http://localhost:8000/v1/metrics/harper/ui

Use the portal to inspect:

  1. The model used for each phase.
  2. Duration and token usage.
  3. Whether /eval passed or failed.
  4. Which test or lint case failed.
  5. Whether /gate promoted or blocked the REQ.
  6. Whether the failure came from code, runtime setup, missing dependencies, or structural gate blockers.

Operational Value

The Telemetry Portal gives CLike a measurable feedback loop.

Without telemetry, AI-native development risks becoming opaque: models generate code, agents execute actions, and developers only see the final output.

With telemetry, every phase becomes measurable:

intent -> generation -> evaluation -> gate -> promotion

This allows the team to improve:


The CLike Harper Telemetry Portal provides a real-time operational view of the AI-native development pipeline. It captures model, token, duration, artifact, eval, and gate data for each Harper phase, enabling teams to compare Cloud and Agent executions, detect failures, control costs, and continuously improve prompt, eval, and gate quality. It turns vibe coding into an auditable, eval-driven engineering workflow.


Before using the portal, verify:

echo "$HARPER_TELEMETRY_DIR"

Start the Gateway and open:

http://localhost:8000/v1/metrics/harper/ui

If the portal shows no data:

  1. Confirm that Harper commands generated telemetry files.
  2. Confirm the Gateway can read HARPER_TELEMETRY_DIR.
  3. Confirm the project ID matches the telemetry records.
  4. Check Gateway logs for file parsing or path errors.

Future Improvements

Recommended next improvements: