> ## Documentation Index > Fetch the complete documentation index at: https://servo.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Hosted observability > Open your organization's dashboard and alerts, see who can change what in them, and register a destination of your own. Every organization gets its own Grafana workspace, with one managed dashboard and three managed alerts. What you see belongs to your organization alone. ## Open your workspace Servo creates the workspace the first time a member asks for it, so there is nothing to install or import. A signed-in member of your organization asks with `GET /v1/observability/workspace`, and [Identity and auth](/guides/identity) covers signing in. The reply carries `status` and `operational_entry_url`, the dashboard address with your organization already selected. Open `operational_entry_url` in a browser once `status` is `ready`, which means Servo has read the dashboard back from Grafana. `provisioning` and `degraded` mean it is being prepared or repaired, so ask again in a moment, and `disabled` means your organization is disabled. An organization admin can have Servo check it again with `POST /v1/observability/workspace/reconcile`. Print a dashboard address under each of your deployments from your computer: ```bash your computer theme={null} servo deployment list --links ``` The command also prints a Rerun address. Use `operational_entry_url` for the dashboard with your organization selected. ## Roles in Grafana Your role in Servo sets your role in Grafana, and Servo keeps the two in step. | In Servo | In Grafana | May | | ------------------ | -------------- | ---------------------------------------------- | | Organization admin | Editor | Build dashboards and alert rules of their own | | Member | Read-only role | Look at dashboards and alerts, and change none | Neither role is Grafana's Admin, the only role that can edit a data source, so nobody in your organization can repoint one at another organization's data. Servo reads your organization's member list when it creates the workspace and every five minutes after that. It adds a new member and removes a departed one as it happens, and it re-checks a member each time they ask for the workspace. A person you remove loses access at the change or the next read. A changed role reaches Grafana at the next check or when that member next asks for the workspace, not at the change. ## The performance dashboard The read-only dashboard in Servo Managed shows action timing across your deployments. It carries the `servo-managed` tag, covers six hours, and refreshes every five minutes. | Panel | Answers | | ------------------------------- | --------------------------------------------------------------------------------------------------------- | | Actions and Errors | How many action requests the endpoint processed, and how many ended in an error | | Server and Model p50 and p95 | How long the endpoint took per action all told, and how long the model itself took | | H264 decode p50 and p95 | How long camera decoding took, for cameras sent as H.264 | | Endpoint round trip p50 and p95 | How long the client waited on the network and endpoint together | | Time breakdowns at p50 | Server: queue, model, camera decode, and other work. Client: capture, encode, request, wire, and response | | Server p95 latency over time | Whether the slow tail is drifting | | What ran | Which model, GPU, host, data type, compile mode, and embodiment made the actions | The server total already includes queue, decode, and model time, so do not add nested stages together. The dashboard is built from metrics only. The workspace also has Tempo and Loki data sources for traces and logs, and a log line that carries a trace id links to its trace. Robot metadata comes from the authenticated inventory API: stable ID, name, type, site, labels, readiness, and last-seen state. Logs and traces use `robot.id` to scope robot views; Prometheus metrics do not repeat robot IDs. Servo re-applies this dashboard and the three alert rules each time it checks your workspace, about every five minutes, so an edit to them does not last. Servo never changes a dashboard or rule it did not create, so build your own beside them. ## Managed alerts Three alert rules sit beside the dashboard, and all are advisory: they show state in Grafana and never change what a robot does. | Alert | Fires when | | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | Servo — control deadline margin exhausted | The margin between queued actions and the wait for the next chunk is below zero in the worst five percent of chunks over the last five minutes | | Servo — released action missing controller receipt | More actions were released than controller receipts were recorded over the last five minutes | | Servo — OTLP ingestion appears dark | No spans reached the workspace for ten minutes, so the other alerts have nothing to read | Each rule must hold for one minute before it fires. A rule with no data counts as firing, so a rule shows firing until the signal it reads first arrives. Servo creates the rules and does not configure where a firing alert is sent. `GET /v1/observability/views` lists the dashboard and the deadline and receipt alerts with their versions and the question each answers. ## How isolation works Isolation is enforced where the data lives, not by a filter in a dashboard. * Each Servo organization has its own Grafana organization, and its data sources send your organization's identity to the log, trace, and metric stores. * Telemetry arrives with a write-only token for your organization. A gateway checks the token and sets the tenant itself, so a sender cannot name another organization. * Your workspace's members are mirrored from your organization's member list. The only other members are Servo's own administrator accounts, which are not on that list and are kept. * When an organization is disabled, Servo removes every member from its workspace. * The hosted API takes your organization from your sign-in, never from the request, and an object owned by another organization looks the same as one that does not exist. ## Register your own destination An organization admin can register an OpenTelemetry destination of your own beside the hosted workspace. Registering records the destination and does not change where your robots or endpoints send telemetry. * `POST /v1/observability/destinations` registers an HTTPS endpoint with a credential that Servo stores write-only. Listing shows the endpoint origin and status, never the credential. * `PUT /v1/observability/destinations/{id}` repoints or rotates it, resets verification to `pending`, and revokes the old credential when you supply a new one. It also re-enables a disabled destination, which needs a new credential. * `POST /v1/observability/destinations/{id}/verify` reports `verified` only when Servo can query a unique probe back from the destination. No query-back check is wired in, so it reports `pending` with the code `query_back_not_configured`, and it refuses a disabled destination. * `DELETE /v1/observability/destinations/{id}` disables the destination and revokes its credential. * Admins read every change in the audit log at `GET /v1/observability/audit`, which never holds a credential. A robot process that connects with an endpoint grant exports to the route in that grant, ahead of any OpenTelemetry settings already in its environment. A worker that Servo starts is given the hosted route. `SERVO_TELEMETRY_PROFILE=off` turns a process's Servo export off. ## You now have * A workspace for your organization, and a role in it that follows your role in Servo. * A dashboard that shows where each action spent its time, and alerts for late control, missing receipts, and silent telemetry. * The rules that keep your data apart from every other organization's, and an admin route to register a destination of your own. ## Next Sign in as yourself, and grant lab-mates a role on a robot. Preflight model inputs, then run a customer-controlled episode.