Skip to main content

Feature status

A snapshot of what's live today versus what's coming. avuru obs v0.18.0 is released and the trunk moves fast — for the plan see the Roadmap, and for shipped changes the Changelog.

Available now​

CapabilityStatusSince
Cluster X-Ray — interactive Node/Pod placement, transparent layers and trace-backed Pod connections, with inventory fallback and explicit scene limitsShippedv0.18
Multiservice log explorer — several services and workloads as one merged stream or panels per service, application/ztunnel/waypoint/other sources chosen independently, from Signals → Logs and Service Mesh → Logs; composed GET /api/v1/logs with a signed pagination tokenShippedv0.19
Service map — call edges derived from traces, health-status rings from the service-health rollup, per-edge caller-side p50/p95 latency, hover-to-focus, and shareable search/status/group filtersShippedM1
Virtual targets — the databases, caches and message brokers your services depend on, derived from their exit spans; no agent in the dependency, and a broker drawn from both endsShippedv0.8
Map boundaries — group the graph by Kubernetes namespace or by service group, each drawn as a labelled containerShippedv0.8
Undetected peers — the far end of a connection nobody instrumented, drawn instead of discarded, and counted apart from servicesShippedv0.8
Service neighbourhood — a service's callers and dependencies drawn as a fixed left-to-right diagram on its Overview tab, each arrow labelled with that path's rate and caller-side p95. Same edges as the tables, no extra request, and the same refusals: a hop recovered across a mesh proxy reads via <proxy>, an untimed edge carries no latency rather than 0ms, and a peer that never sent a span is outlined rather than filledShippedv0.13
Focus on the map — "Show on the map" keeps a service and its one-hop neighbourhood instead of filtering the graph down to an isolated nodeShippedv0.13
Traces — search, filters (tags, order, duration, status), latency heatmap, per-operation RED overviewShippedM1
Trace viewer — timeline, spans table, flamegraph, statistics, trace graph, JSONShippedM4
Trace comparison — structural diff of two tracesShippedM4
Logs — full-text search, severity/service filters, trace_id correlationShippedM1
System Status — component health, storage, retention, diskShippedM1
Sensor DaemonSet — zero-code eBPF traces + RED (OBI), zero-config log collectionShippedM2
Service inventory — sortable RED table, drill-down to tracesShippedM2
RED metrics dashboard — rate/errors/duration charts per serviceShippedM3
Node / pod health — CPU, memory, network per node & pod (kubeletstats), sortable on every column and filterable by name, namespace or workloadShippedM3
Continuous profiling — CPU flame graphs per service (experimental, opt-in)ShippedM4
Projects — per-environment scoping, sidebar switcher, shareable ?project= linksShippedv0.1
Project management — create, rename and delete projects from the UI; default and config-defined projects stay read-onlyShippedv0.3
Read-only demo — one-click "Try the demo" viewer backed by live data (OpenTelemetry Astronomy Shop); opt-in, server-side sign-inShippedv0.3
Member projects — one project reads the union of several clusters on every screen; membership is one level deep and each viewer still sees only the members they were grantedShippedv0.6
Per-project retention — give a project a shorter window than the install, enforced by an hourly scoped trimShippedv0.6
Per-project storage usage — rows, estimated size, ingest rate and effective retention for the selected project, beside the instance-wide totalsShippedv0.6
Component toggles — install the ingest half alone on a secondary cluster (hub.enabled / ui.enabled / gateway.enabled), writing to a shared storeShippedv0.6
Collection controls — per signal/namespace/pod/node, with a per-node agent inventoryShippedv0.1
Runtime collection control — switch each signal on or off from Settings → Collection and the sensor follows in seconds, no helm upgrade; shows the effective config and resets to chart defaults. Opt-in, behind a namespace-scoped Role over the hub's own sensor resourcesShippedv0.5
Error tracking — deduplicated issues from spans/logs, triage lifecycle, Sentry-protocol ingestShippedv0.2
Service health — group health with criticality tiers and critical-dependency propagation, derived from REDShippedv0.2
Service groups from the UI — create, edit and delete health groups in Settings → Groups; applies to the next health read, no helm upgrade. Chart-declared groups stay read-only and win a name collision; auto-grouping by namespace is unchangedShippedv0.5
Alerting — webhook notifications when a service/group crosses into a bad stateShippedv0.2
Network health on the service map — per-edge RTT + failed/reset connections (OBI TCP stats)Shippedv0.2
Green — per-service energy (Wh) & carbon (gCO2e) via CNCF Kepler, carbon budgets, CSRD-ready export (off by default; measured on RAPL hardware)Shippedv0.2
Sensor safety — CI-proven "do no harm" gate (probe-sensitive canary) + opt-in instrumentation mode (discovery.mode)Shippedv0.2
Authentication — secure by default: local users, fixed roles (admin/editor/viewer) with per-project grants, server-side sessions, Settings → Users, opt-in anonymous viewerShippedv0.2
SSO — OpenID Connect (discovery, PKCE) with any IdP, group→role/project mapping (hot-reloaded), forceSSOShippedv0.2
Modules — one switch per signal family (modules.<name>.enabled) gates schema, API, pipeline, collection and UI together; capabilities-driven sidebarShippedv0.2
Ingest API keys — per-project keys validated in the gateway; enforce makes the key's project the authoritative tenant; safe off/log/enforce rolloutShippedv0.3
Green on RAPL-less nodes — opt-in TDP power estimation for cloud VMs, labeled estimated end to end and never blended with measured energy; /green coverage panel. The measured source can be dropped entirely (sensor.green.kepler.enabled=false) so a RAPL-less fleet keeps its other signalsShippedv0.3
User management — edit a user's name and role grants, reset passwords, and delete users behind a disable-first rule; password operations refused for SSO accountsShippedv0.4
Self-service password change — Settings → Account, current password required, other sessions evicted while yours stays liveShippedv0.4
Schema self-healing — the hub applies missing migrations on connect (hub.autoMigrate) and reports applied vs. expected as a Schema component in Settings → StatusShippedv0.4
Reverse-proxy support for login — auth.trustedOrigins plus an enforce/log/off origin check for proxies that rewrite HostShippedv0.4
Storage view — ClickHouse connection (read-only), per-signal size, compression and age, and configured retention shown against the TTL the tables actually enforceShippedv0.5
Access view — which role may read and which may change each area, derived by the hub from the guards its routes registered with, so it cannot drift from what is enforcedShippedv0.5
Dashboard — the landing screen: service-group health, compact topology, firing alerts and Kubernetes capacity in one view. Each band follows its module, and falls back to the busiest services rather than disappearing when service health is offShippedv0.5
SSO group mapping from the UI — Settings → Access shows the chart-declared OIDC group→role rules and lets an admin author, edit and delete rules beside them; the chart wins a name collision and an overridden rule says why. A change applies on the group's next sign-in or token refresh, cluster-wide within ~15 sShippedv0.5
Personal API tokens — mint, list and revoke bearer tokens in Settings → Access; only the SHA-256 is stored and the raw value is shown once. A token resolves to its owner's live grants, so disabling a user disables every token they hold; a bad token is a clean 401, never an anonymous downgradeShippedv0.5
Wider ingest compatibility — Jaeger, Zipkin, Prometheus remote-write and Loki push receivers alongside OTLP, one values flag each and all off by default; every receiver goes through the same tenant stage, so ingest keys are enforced identically whatever the wire protocol. Forwarding exporters (OTLP/Kafka) dual-write to a second backend during a migration, behind a bounded queue so a dead target cannot backpressure storageShippedv0.6
Green coverage and budget deliverability — a per-node energy table under /green with the measured/estimated split kept per row, carbon budgets that say whether they can actually reach a channel, and a warning when a budget targets a service group nothing rolls up toShippedv0.6
Business tags — map a Kubernetes pod label once (tags.labels) and it rides every signal as avuru.tag.<key>, applied at collection so uninstrumented workloads carry it too; then filter traces and logs by it, with the discovered keys and values offered as controls. A trace matches when any service that took part carries the tagShippedv0.7
Declared service metadata — a service states its own domain (service.namespace), environment and avuru.tier as resource attributes and the health board groups it accordingly, across Kubernetes namespaces, with no hub config. Operator config still wins, and a declaration the hub cannot use is surfaced rather than silently droppedShippedv0.7
avuruobs CLI — services, health, traces, logs, status over the public API with a personal token, -o table|json, and a --fail-on predicate for CI gates with three exit codes so a tripped gate is distinguishable from a broken oneShippedv0.7
Grafana data source — service RED, service health, trace search and cross-zone traffic in dashboards you already run. A backend plugin, so the API token never reaches a browser and queries leave the Grafana serverShippedv0.7
Inter-zone traffic accounting — bytes per availability-zone pair from kernel flows, standalone from the per-edge network feature so zone accounting costs zone-pair cardinality rather than workload-pairShippedv0.7
Mesh-aware service map — transport workloads (mesh sidecars, waypoint and ztunnel proxies, ingress and egress gateways) are recognised and hidden behind a Show mesh & gateways toggle instead of being drawn as dependencies; flow-derived edges are drawn and counted apart from traced callsShippedv0.7
Contextual documentation — every screen links to the page of the manual that explains itShippedv0.7
A richer service map — namespace and service-group boundaries, edge volume on every edge on demand, undetected peers drawn instead of discarded, a legend that explains every channel in use, and a zoom readoutShippedv0.8
Virtual targets — databases, caches and message brokers as first-class map nodes, derived from the exit spans already stored. A broker is drawn from both ends, so a queue is never a dead endShippedv0.8
Layered navigation — the sidebar grouped by the question each screen answers (Topology, Signals, Operations, Infrastructure), with the first-five-minutes path unchanged and a layer whose every screen is inactive disappearing rather than labelling a gapShippedv0.8
Transport hop collapse — the real app → app dependency recovered across up to three chained mesh proxies by walking each trace's own ancestry, named with the proxy it came through. The Show mesh & gateways toggle swaps representations rather than stacking them, so one request is never drawn twiceShippedv0.9
Mesh surfaces — per-proxy load, latency and success rate with calls carried in and out counted apart, plus control-plane health: connected proxies, push convergence, and the configuration your proxies refused. With nothing scraped the screen says so rather than reporting zeroShippedv0.9
TCP retransmits on the map's edges — packet loss on a link, which RTT alone cannot reveal, alongside RTT p95 and failed connections; an edge that retransmits is styled unhealthy on its own accountShippedv0.9
Per-edge network attribution, asserted on a live eBPF cluster in CI rather than assumed — which uncovered and fixed a latent crash that took the whole sensor down whenever TCP stats were enabledShippedv0.9
Endpoint checks — scheduled probes attached to a service-health group, so a group with no traffic can be told apart from a group with no service. Two consecutive failures move a group, never one, and each probe emits a span of its own so a failing check links to the trace of the request that failedShippedv0.9
Cost & waste — every workload's reserved CPU and memory against what it actually used, ranked by the gap, with workloads that declare no request called out as their own state and node allocation shown beside node usage. Idle is measured against the peak, never the mean. Rates are yours to declare; with none set the screens report cores and bytes and say so — there is no pricing API and no egress. Off by defaultShippedv0.10
Transport recognised from Kubernetes labels — a gateway you named anything at all is classified from the label your mesh wrote on it, with names still the answer for sidecars (which wear their application's labels) and your applications override still finalShippedv0.10
Control-plane silence explained — nothing scraping, target not answering, or answered with metrics that are not a control plane this product reads. Three states with three different fixes, where there used to be one sentenceShippedv0.10
AI observability — the model calls your applications already send: per model, calls, tokens in and out, latency, failures and truncation; per calling service, the same with an owner. The model that answered wins over the one requested, both token spellings are read, a call that reported no usage is excluded rather than counted as zero, and truncation is reported apart from failure. Prices are yours to declare; there is no pricing API. Off by defaultShippedv0.11
Prompt and completion text dropped at the gateway by default — message content reaches storage only because an application's SDK captured it, and until now it was stored under the ordinary retention and shown to every Viewer. Anchored so a token count is never mistaken for a prompt, and deliberately not gated on the AI moduleShippedv0.11
Trace breakdown — the traffic as a treemap and a donut, grouped by service, operation, outcome, span kind or any span/resource attribute, weighted by request count or total time, over an explicit span population (served / entered / every span). The tail is a real bucket, so the parts sum to the wholeShippedv0.11
Service detail page — one service's health, RED over time, callers and dependencies as two separate lists, and its traces, logs and error issues behind tabs. Composed from reads that already existedShippedv0.11
Trace path — the service-level graph of a single request, weighted by time spent inside each service rather than span duration, with dependencies that never reported drawn as the terminal hops they areShippedv0.11
Refused — server-side 4xx as its own outcome beside ok and error, on the span badge, the operations overview, the trace table and the search filter, and deliberately kept out of the error rateShippedv0.11
Tool calls told apart from model calls — the AI module tested that gen_ai.operation.name was present and never read its value, so on an agent workload every execute_tool span was counted as a call to a model: call counts inflated, latency mixed a database lookup with a completion, the model resolved to nothing, and the no-usage bucket filled with spans that were never model calls. Embeddings stay counted as model calls, because they spend real tokensShippedv0.12
Tools table — per tool an agent ran: calls, failures, p95 and the services invoking it, with a tool hit four times shown as one row with a count. No tokens and no cost columns, because a tool execution spends neither and a zero would read as freeShippedv0.12
Agent turn view — one turn drawn as the graph it is, model calls and tool executions as distinct node kinds, weighted by time spent inside each call rather than span duration. Offered only on traces that hold a turnShippedv0.12
Spend budgets — monthly ceilings in tokens or money, per calling service or across the estate, firing through the alerting channels you already have. A money budget with no prices declared is refused at startup rather than sitting under every threshold forever, and a partly priced scope says its figure is a floorShippedv0.12
One rate table — model prices and compute rates in a single document with a single currency, editable in Settings and applied without a redeploy. Chart-declared values stay readable and read-only, UI entries overlay them, and every row says which it isShippedv0.12
MCP server — six read-only tools over the traces, logs, error issues and health you already store, so an agent can investigate an incident instead of a person retyping a screen. One handler on the hub, authenticated with personal API tokens resolving their owner's live permissions. A misspelled service returns the closest matches rather than an empty result, and a section this install cannot answer is named rather than silently missing. Off by default: what an agent reads leaves your cluster for the model provider you chose, so the switch is yours — and every tool call is logged with the token owner, the tool, its arguments and the row countShippedv0.12
Collector images without a fixable Critical — the gateway distro pins x/crypto, x/text and grpc above their advisories, and the node agent tracks the collector line that scans clearest. A registry with a block-on-critical policy stops serving a flagged image, which surfaces as a rollout that times out rather than as a scan reportShippedv0.12
No image needs a registry allowlist — the node agent gets a first-party collector distro of its own, carrying exactly the components its rendered config uses and pinning x/crypto and x/mod where no collector release does. Every image the chart pulls by default scans free of a fixable Critical or High; the stock upstream image stays available as an override, and the values file says what that costsShippedv0.13
A hosted assistant can connect — the MCP server speaks OAuth 2.1, so a client you do not run yourself can sign in: discovery, registration, authorization code with PKCE, rotating refresh tokens. Access tokens are opaque and bound to the MCP endpoint, so one can never be replayed against the rest of the API, and what it may reach is re-read on every request. The consent screen states that approving sends traces and log bodies out of the installation, marks the application's self-declared name as unverified, and limits access to one project; Settings → Access lists what you have connected and disconnects it. Off by default, behind its own switchShippedv0.14
Mesh proxies by role — control plane, ingress and egress gateway, waypoint, ztunnel and sidecar as filterable roles beside the namespace each runs in, read from labels your mesh already writes and storage was already keeping. Calls carried in and out are named as calls; bytes, round-trip time, failed connections and retransmits get columns of their own, and an install that measures none of them shows no column rather than a zeroShippedv0.14
A proxy you can open — one proxy's own rate, errors and latency over time, and what it carries: the real app → app dependencies recovered through it with the number of proxies each crossed. Plus a graph of the mesh drawn with those hops left in, which is what the service map exists to take out. Composed from reads that already existedShippedv0.14
The mesh's own configuration, and what is wrong with it — a separately granted, read-only module (get/list/watch, its own service account, no write verb the chart will render) that lists every namespace the cluster defines, including the ones enrolled and silent that no telemetry could ever show, and checks six ways a mesh breaks without emitting anything: a route pointing at a service or port that does not exist, a route naming an absent gateway, a gateway nothing attaches to, an unresolved host, TLS disabled under a strict rule, a workload sent to a waypoint that is not there. Off by default, and separate from the mesh module precisely so the cluster-wide read stays an explicit decisionShippedv0.14
The proxies' own account of their traffic — each node's sensor scrapes the sidecars, waypoints, gateways and ztunnel on that node, discovered from annotations the mesh already writes, so there is no endpoint to type and every proxy is reached exactly once: mutual TLS or plaintext per request and per connection, the response flag a failure carried, the destination version, ztunnel's own count of the workloads it carries, and up per proxy so nobody is scraping reads differently from the proxies are not answering. On by default under the mesh module (mesh.dataPlane.enabled=false to keep the screen without it); installs without the module are untouchedShippedv0.15
Declared beside observed — a Security tab with one row per workload: the PeerAuthentication mode in force with the scope that decided it, the observed mutual-TLS share, and one of four verdicts — strict and all mTLS; declared strict, observed plaintext (a finding: the policy is not applied); permissive but safe to tighten; permissive with plaintext callers, named. Traffic in an ambient namespace that no proxy carried is its own finding. No percentage until the data plane was actually read, and no Declared column when the configuration module is offShippedv0.15
Every workload the cluster runs, in the mesh or not — a Workloads tab reading pods (still get/list/watch, a dozen fields kept per pod, capped on their own): captured, sidecar, declared, not enrolled or out of mesh, with the waypoint that binds it and why, the covering policies, the declared mTLS mode with the policy that decided it (selector-scoped included), the observed share, its traffic and its findings. A page per workload with its pods; a waypoint page listing what it serves and whether anything runs it; namespace rows saying where their mode came from and how many workloads are enrolledShippedv0.15
Seventeen configuration checks — v0.14's six plus the ones aimed at configuration that looks finished and is not: a workload labelled for ambient never captured, a policy matching no pod or naming nothing, a binding to a waypoint nobody deployed, HTTP-level rules in an ambient namespace with no waypoint, a sidecar in an ambient namespace, a route to an undefined subset, two rules claiming one host, a gateway no pod serves, listeners that conflict, an authorization rule naming a service account nobody runs as; the mTLS conflict judged per workload, in both directions. The five that need pods go silent and say so when the pod list was refused or cutShippedv0.15
A proxy's requests by outcome — by response flag with the proxy's reason in words (circuit breaker open, retries exhausted, upstream would not connect, no route), by destination version, and by caller with its 5xx count. Per-upstream counters a default mesh does not expose are named as not collected, with the setting that exposes them, never rendered as zero. Ztunnel rows carry workloads carried and still waiting; the control-plane card adds listener conflicts and queue p95Shippedv0.15
The mesh graph tells its proxies apart — a star for the control plane, a tag for a gateway pointing in and its mirror for one pointing out, a chevron for ztunnel, a double ring for a waypoint, with a legend naming only the shapes on it — and every edge a destination proxy measured carries a marker at the caller end, on the service map and the mesh graph alike: a tee for all mutual TLS, a hollow circle for mixed, a filled one for plaintext. An edge nobody measured carries noneShippedv0.15
A workload's page reads like the cluster's record of it — created when and by which controller (or by its oldest pod, and it says which), type, app and version, every label, the controller's annotations within stated bounds, each pod with the rollout it belongs to, and a one-word health verdict with the reason that decided it. Beside the policies that select the workload by label, the routes and rules that reach it through its Services — HTTPRoute, GRPCRoute, VirtualService, DestinationRule — each with its own findings, so a routed workload is never called unconfigured. Nothing new is read: the objects were already watchedShippedv0.16
A workload's logs from all three sources, in one table — its own lines, the ztunnel lines naming one of its pods and the waypoint lines naming its Service, as one ordered stream under one cursor, composed by the hub that knows the pods. Search, minimum severity and a checkbox per source in the URL, the workload's own lines by default and the proxies' a click away; a line saying what was actually asked; and when the pods cannot be known, matching by name and namespace together with the reason stated rather than an empty column. Needs the logs module; mesh-config only makes it preciseShippedv0.16

| Container logs carry a level — the node agent reads it off each tailed line (a JSON level field, named or numeric; a level= pair; the usual upper-case token) into the OTel severity, so severity filters and error issues from logs work for apps that only write to stdout; records that already carry a level are left alone. Off with sensor.agent.logs.parseSeverity=false | Shipped | v0.17 | | The Errors screen says what its issue list adds up to — issues matching, how many were first seen in the window, how many regressed, the occurrences they produced with a histogram, and the busiest services as one-click filters. The band aggregates the same issue set the rows below it query, through the same SQL, so the two cannot state different totals. The list also scrolls now, with a sticky header, and names the real total when a filter matches more than the 200 it loads | Shipped | v0.17 | | A service's logs, found under whatever name its pod files them — a service's Logs tab ties the service to its workload (its spans' k8s.deployment.name, or the Kubernetes Service in front of it) and reads the same three sources as the workload page: its own lines under either name, and ztunnel's and the waypoint's a checkbox away. When no workload can be tied to it, the tab shows the service's own lines and one sentence naming what to fix. Needs only the logs module; the proxy sources need mesh-config | Shipped | v0.17 | | Logs you can take out of the screen — every log table copies all loaded lines, or a selection made with shift-click ranges, and downloads what is loaded as a .log file, each control naming how many lines it will take; the per-row copy button is reachable by keyboard | Shipped | v0.17 | | Explorer opens on the connections between services — the service map is the application entry point; selecting a node keeps the map visible and opens an inspector with callers, dependencies and links to service details, traces and logs; the selection is shareable in the URL and reachable from the keyboard, and an empty project explains how to connect eBPF and OTLP | Shipped | v0.17 | | Cluster X-Ray — an interactive isometric view of Nodes, translucent Pods and a logical infrastructure layer beside the inventory: orbit, zoom, separate or hide layers, filter namespaces, inspect a Pod's CPU, memory and placement, and follow the connections your traces recorded (GET /api/v1/infra/pod-connections: caller request counts, errors and p95, only between recorded Pod identities). Loads on demand, states its scene limits, and the inventory stays for keyboards and browsers without WebGL | Shipped | v0.18 | | One log explorer for several services and workloads — pick them, choose which sources to read (the applications' own lines, ztunnel's, the waypoints', everything else) and follow them as one merged newest-first stream or as one panel per service, from Signals → Logs and from Service Mesh → Logs; selection, sources and display live in the URL, and GET /api/v1/logs composes the subjects in one paginated query under a signed pagination token | Shipped | v0.19 | | Slate dark theme — slate surfaces and soft blue actions replace forest and lime in dark mode; the service map and Cluster X-Ray follow, the light theme is unchanged, green means healthy only, and secondary text is a real token that clears 4.5:1 on every surface | Shipped | v0.19 | | Service Mesh shows a project-scoped account only its own projects' namespaces — the namespace list, workload list, configuration browser, workload and waypoint detail are narrowed to the namespaces the caller's projects reach, derived from that project's telemetry in the window; an identity that may see every project still sees the cluster whole | Shipped | v0.19.1 |

Coming​

CapabilityStatusTarget
Control planes other than Istio — the per-proxy half already works anywhere; the control-plane half is Istio-shaped todayPlannednext
Cost joined to green — the same reserved-and-idle capacity in Wh and gCO2e, on an install running bothPlannednext
Typed decisions — a classifier that answers with a calibrated probability, never text, for the severity a rule could not read, errors a fingerprint kept apart and alerts to triage; at most an opt-in, batch module sending structured or scrubbed payloads, measured by an offline harness before any module exists (design note)Plannedunder evaluation

:::info Service map today Topology edges are derived from trace spans (cross-service Client/Server pairs) — and with the sensor DaemonSet, every HTTP/gRPC service is traced zero-code, so uninstrumented apps appear too. OBI's built-in network feature further enriches the map with un-instrumented edges and per-edge network health when enabled. A node's ring is read from the service-health rollup, not re-derived on the map — a service outside that rollup's coverage reads unknown, never healthy. :::