# Observatory Agent Guide Observatory is a multi-user cost and operational-metrics app. Each signed-in user has an isolated set of provider connectors. The dashboard is read-only; a user's agent configures connectors through the authenticated API. ## API - `GET /api/connectors` — connector catalog and the current user's connection status. - `PUT /api/connectors/{provider}` — create or replace one connector. - `DELETE /api/connectors/{provider}` — disconnect one provider. - `GET /api/series?days=30` — authenticated canonical provider envelopes. The AWS envelope also includes the current regional EC2 instance inventory, state summary, and an explicit inventory error when EC2 read access is unavailable. - `GET /api/series/{provider}?days=30` — query one provider independently. - `GET /api/projection?days=30` — generate or reuse a clearly labeled AI projection for the incomplete current-day cost bucket. Observed values remain separate and act as a floor. - `POST /api/metrics/query` — run a bounded AWS CloudWatch metric query. - `GET /api/me` — authenticated viewer identity. - `GET /api/health` and `GET /api/version` — public operational checks. Connector configuration is encrypted at rest, scoped to the authenticated user, and never returned by the API. An agent may retrieve credentials from the user's preferred secrets manager, then send them directly over HTTPS; Observatory does not need access to that secrets manager. `PUT` is idempotent and replaces the previous credential set. Authenticated calls accept either the user's AuthReturn JWT or that user's app-specific AuthReturn API key in `Authorization: Bearer …`. Agents should use an API key for unattended access; keys remain user-scoped and revocable. ## Connect a provider 1. Ask the user which provider to connect and which secrets-manager entry holds its dedicated billing credential. Read the secret directly from that manager; do not ask the user to paste it into chat. 2. Authenticate as that same Observatory user. For an interactive, bounded setup, use the user's Observatory AuthReturn JWT. For unattended refreshes, use that user's app-specific, revocable Observatory API key. Never use another user's bearer credential. 3. Call `GET /api/connectors` and select one of the provider IDs below. Send the exact required fields in a `PUT /api/connectors/{provider}` request over HTTPS. 4. Confirm that the response reports `connected: true`, then call `GET /api/series/{provider}?days=30`. Report provider permission errors as-is; do not broaden the credential automatically. For a bounded setup, obtain the JWT from the canonical app-scoped login endpoint. Read the email and password from the user's secrets manager and keep the returned token out of logs: ```http POST https://authreturn.com/api/apps/observatory/login Content-Type: application/json {"email":"","password":""} ``` Use the response's `token` only as `Authorization: Bearer ` on Observatory requests. Authentication must terminate with a visible error within 20 seconds; do not retry invalid credentials. For unattended use, store a user-scoped Observatory API key in the user's secrets manager and revoke it when the agent no longer needs access. Example request: ```http PUT /api/connectors/openai Authorization: Bearer Content-Type: application/json {"config":{"admin_key":""}} ``` The authenticated connector endpoints are on `https://observatory.aisloppy.com`. Apply an explicit timeout to every request; 30 seconds is suitable for connector configuration and 45 seconds for the first provider query. The API never returns the submitted credential. Supported provider configuration: | Provider | Required fields | Optional fields | |---|---|---| | `aws` | `access_key_id`, `secret_access_key` | `session_token`, `region` | | `gcp` | `project_id`, `dataset_id`, `table_id`, `service_account` object | — | | `openai` | `admin_key` | — | | `anthropic` | `admin_key` | — | | `scrapingbee` | `api_key` | — | ## Connect AWS costs and CloudWatch The existing `aws` connector powers both Cost Explorer and CloudWatch. Use a dedicated IAM principal with this minimum read-only policy; scope resources further when the account's metric naming allows it: ```json {"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["ce:GetCostAndUsage","cloudwatch:GetMetricStatistics","cloudwatch:ListMetrics"],"Resource":"*"}]} ``` The metrics endpoint accepts JSON containing `namespace`, `metric_name`, an optional `dimensions` object, `statistic`, `period`, and `hours`. Queries are limited to 30 days, 1,440 points, ten dimensions, known CloudWatch statistics, and a 20-second overall deadline. Successful identical queries are cached for 60 seconds. Example: ```http POST /api/metrics/query Authorization: Bearer Content-Type: application/json {"namespace":"AWS/EC2","metric_name":"CPUUtilization","dimensions":{"InstanceId":"i-0123456789abcdef0"},"statistic":"Average","period":300,"hours":24} ``` CloudWatch errors are visible but connector credentials are redacted. The API returns timestamp/value points and never returns the AWS credential. Provider-specific payload shapes: ```json {"config":{"access_key_id":"","secret_access_key":"","region":"us-east-1"}} {"config":{"project_id":"","dataset_id":"","table_id":"","service_account":{"type":"service_account","project_id":"","private_key_id":"","private_key":"","client_email":"","client_id":"","token_uri":"https://oauth2.googleapis.com/token"}}} {"config":{"admin_key":""}} {"config":{"admin_key":""}} {"config":{"api_key":""}} ``` These examples are ordered as AWS, GCP, OpenAI, Anthropic, and ScrapingBee. Send only the payload for the selected provider. Optional AWS `session_token` credentials expire; replace the connector before expiry or use a dedicated long-lived read-only IAM principal. Use read-only billing or usage credentials with the narrowest provider permissions available. Do not put credentials in query strings, logs, chat messages, or source control. After configuration, query `/api/series/{provider}` to verify access; a failed provider returns a visible error without affecting other connectors. ## Security model - Public signup is enabled. Every connector, provider cache entry, and usage-history path is scoped by the verified AuthReturn user ID. - Connector secrets terminate at Observatory: the service must decrypt them in memory to call providers. This is encrypted-at-rest isolation, not a zero-knowledge vault. Host administrators retain infrastructure-level access. - Credentials are never returned after configuration. Sensitive values are redacted from upstream errors, protected API responses are `no-store`, and rendered provider content uses text nodes rather than HTML injection. - Browser sessions require an AuthReturn Cognito ID token with verified issuer, audience, signature, expiry, and token type. Automation keys are verified against the `observatory` AuthReturn app and successful verification is cached for at most 60 seconds. - Use a dedicated read-only billing identity per provider. Do not supply general cloud-administrator, resource-write, or payment-method credentials. Minimum credential guidance: | Provider | Recommended boundary | Residual power | |---|---|---| | AWS | A dedicated IAM principal allowing `ce:GetCostAndUsage`, `cloudwatch:GetMetricStatistics`, and `cloudwatch:ListMetrics` | Do not reuse an infrastructure automation key. | | GCP | BigQuery Job User plus Data Viewer limited to the billing-export dataset | The service-account key remains a reusable bearer credential. | | OpenAI | A dedicated organization admin key used only for usage and costs | OpenAI's usage/cost endpoints require organization-level admin authority. | | Anthropic | A dedicated organization admin key | The usage endpoint requires admin authority. | | ScrapingBee | A dedicated account key | ScrapingBee keys can make paid scraping requests; no read-only usage key is assumed. | Observatory currently accepts encrypted inline credentials; it does not yet resolve references from an external user-controlled vault or use provider OAuth. ## Connector adoption analytics Observatory is registered with Grid Glance as `observatory`. The backend emits these privacy-safe events: - `connection_attempted`: an authenticated connector `PUT` reached the backend. This is the primary adoption KPI. - `connection_succeeded`: connector configuration passed validation and was encrypted and stored. It does not claim that the upstream provider accepted it. - `connection_failed`: the request ended in one of `invalid_request`, `unknown_provider`, `invalid_config`, or `storage_error`. - `guide_opened`: a signed-in user followed a disconnected card's setup link. Event paths contain only the event, a catalog provider ID, and the controlled failure category. Attempt sessions use an app-secret HMAC of the AuthReturn user ID so Grid Glance can count unique attempting tenants without receiving the raw ID or email. Connector configs, API keys, secrets, raw errors, request bodies, and email addresses are never analytics fields. View the `observatory` panel at `https://gridglance.com/static/dashboard.html`. Attempt totals and provider breakdown are the page counts under `/events/connection_attempted/{provider}`; attempt-only session analytics provide the unique-tenant count. No separate Observatory analytics dashboard is maintained. The cost-series response is schema version 2. AWS, GCP, OpenAI, Anthropic, and ScrapingBee return independent availability envelopes; missing or failed providers are never silently replaced with zeroes. Windows are bounded to 2–366 days. The dashboard includes a stacked daily cost graph, provider totals, token usage, service/model breakdowns, and ScrapingBee credit headroom. The final chart date may use a BrightWrapper projection because provider billing buckets publish at different times. Projection payloads are labeled `kind: ai_projection`, retain each provider's `observed_usd`, and never change provider totals or cached source observations. Recent non-read-only CloudTrail events are weak context when the connected AWS identity permits lookup; an unavailable CloudTrail query is explicit and does not block history-based estimates. Base URL: `https://observatory.aisloppy.com` ## Purpose User-isolated cost, usage, and operational metric tracking for connected services ## Standard Endpoints - `GET /agent-guide.md` - Machine-readable integration guide. - `GET /agent-guide` - Human-readable rendered guide. ## Authentication - Endpoints marked as auth-protected require a valid JWT. - Browser clients should use the app's own sign-in UI. - API clients should send `Authorization: Bearer ` unless the app guide documents a different auth mechanism. - If the app exposes local signup or login endpoints, document those app-local endpoints here instead of pointing agents at third-party auth vendors. ## Integration Rules - Fail fast on errors; do not silently degrade. - Read `PORT` from environment. - Use network APIs between services; no cross-app imports. - Long-running work must start async jobs and return `202` with a task ID quickly. - Durable task status belongs on `GET /api/tasks/`. - SSE is optional live transport only; do not make it the sole source of task state. ## Notes For Coding Agents - Start with `GET /agent-guide.md` for current contract. - Confirm endpoint auth requirements before calling. - Include explicit timeouts and propagate errors.