Cloud or Self-Hosted

Grafana Loki Adapter

Push wide events to Grafana Loki — self-hosted, multi-tenant, or Grafana Cloud — with low-cardinality labels and full JSON querying.

The Loki adapter pushes wide events to Grafana Loki via its push API. It works with a self-hosted single-tenant instance, a multi-tenant deployment, and Grafana Cloud.

Each event is pushed as a JSON log line under a small, low-cardinality label set. That distinction matters in Loki: labels are indexed and billed by cardinality, while the log line is searched at query time. evlog labels only service, environment, and level by default, and leaves everything else (requestId, path, user, your custom fields) in the line, queryable with | json.

Add the Grafana Loki drain adapter

Installation

The Loki adapter comes bundled with evlog:

src/index.ts
import { createLokiDrain } from 'evlog/loki'

Quick Start

Set LOKI_ENDPOINT and wire the drain into your framework.

// server/plugins/evlog.ts
import { createLokiDrain } from 'evlog/loki'

export default defineNitroPlugin((nitroApp) => {
  nitroApp.hooks.hook('evlog:drain', createLokiDrain())
})

Configuration

Environment variables

VariableRequiredDescription
LOKI_ENDPOINTYesBase URL of the Loki instance, without the push path (LOKI_URL also accepted)
LOKI_API_KEYNoAPI token. Sent as Basic with LOKI_USER, otherwise as Bearer (GRAFANA_API_KEY also accepted)
LOKI_USERNoGrafana Cloud instance ID. Switches auth to Basic (GRAFANA_USER also accepted)
LOKI_TENANT_IDNoTenant for multi-tenant self-hosted Loki, sent as X-Scope-OrgID

Priority

Configuration is resolved highest to lowest:

  1. Overrides passed to createLokiDrain()
  2. runtimeConfig.evlog.loki (Nitro)
  3. runtimeConfig.loki (Nitro)
  4. Environment variables

Options

OptionTypeDefaultDescription
endpointstringBase URL, without /loki/api/v1/push
apiKeystringAPI token
userstringGrafana Cloud instance ID (enables Basic auth)
tenantIdstringX-Scope-OrgID for multi-tenant Loki
labelFieldsstring[]['service', 'environment', 'level']Event fields promoted to Loki labels
labelsRecord<string, string>Static labels merged into every stream
timeoutnumber5000Request timeout in ms
retriesnumber2Retry attempts on transient failures

Deployment

Loki runs either way, and the adapter is the same in both cases. Only authentication differs.

Self-hosted

A single-tenant instance needs nothing but the endpoint:

server/plugins/evlog.ts
createLokiDrain({ endpoint: 'http://localhost:3100' })
.env
LOKI_ENDPOINT=http://localhost:3100

For a multi-tenant deployment, name the tenant, and it is sent as X-Scope-OrgID:

server/plugins/evlog.ts
createLokiDrain({
  endpoint: 'http://loki.internal:3100',
  tenantId: 'team-checkout',
})
.env
LOKI_ENDPOINT=http://loki.internal:3100
LOKI_TENANT_ID=team-checkout

If your instance sits behind an authenticating proxy, apiKey alone is sent as Authorization: Bearer.

Grafana Cloud

Grafana Cloud authenticates with your instance ID plus an access policy token, sent together as HTTP Basic:

server/plugins/evlog.ts
createLokiDrain({
  endpoint: 'https://logs-prod-eu-west-0.grafana.net',
  user: '123456',
  apiKey: process.env.GRAFANA_API_KEY,
})
.env
LOKI_ENDPOINT=https://logs-prod-eu-west-0.grafana.net
LOKI_USER=123456
LOKI_API_KEY=glc_xxx
user is the numeric instance ID of the Loki datasource. Find it under Connections → Data sources → Loki in your Grafana Cloud stack. It is not your account email. Using the wrong value is the usual cause of a 401.

Which auth applies

userapiKeytenantIdHeader sent
Authorization: Basic base64(user:apiKey)
Authorization: Bearer <apiKey>
X-Scope-OrgID: <tenantId>
none — unauthenticated instance

tenantId is independent and can be combined with either auth mode.

Labels and cardinality

Never promote a high-cardinality field to a label. Loki creates one stream per unique label combination, so labelling requestId or userId will create millions of streams and degrade or break your instance.

Add a label only for values you filter on and that have a bounded set:

server/plugins/evlog.ts
createLokiDrain({
  // `region` has a handful of values — safe
  labelFields: ['service', 'environment', 'level', 'region'],
  // Static labels for everything in this deployment
  labels: { cluster: 'prod-eu' },
})

Everything else stays in the JSON line and is fully queryable. See below.

Querying in Grafana

Filter by label, then reach into the wide event with | json:

{service="checkout", environment="production"}
  | json
  | status >= 500
# Slow requests on one route
{service="checkout"} | json | path="/api/orders" | durationMs > 1000
# One request end to end
{service="checkout"} | json | requestId="4a8ff3a8-..."
# Error rate per service
sum by (service) (rate({environment="production", level="error"}[5m]))

Batching

Pair the drain with a pipeline to push in batches rather than per request:

server/plugins/evlog.ts
import { createDrainPipeline } from 'evlog/pipeline'
import { createLokiDrain } from 'evlog/loki'
import type { DrainContext } from 'evlog'

const pipeline = createDrainPipeline<DrainContext>({
  batch: { size: 100, intervalMs: 5000 },
})

export default defineNitroPlugin((nitroApp) => {
  nitroApp.hooks.hook('evlog:drain', pipeline(createLokiDrain()))
})

Events sharing a label set are grouped into a single Loki stream, and entries are sorted by timestamp, because Loki rejects out-of-order pushes within a stream.

Verify it locally

Spin up Loki in Docker and push a real event, with no cloud account needed:

Terminal
docker compose -f packages/evlog/test/e2e/docker-compose.yml up -d
LOKI_ENDPOINT=http://localhost:3100 pnpm run test:e2e
docker compose -f packages/evlog/test/e2e/docker-compose.yml down -v

The suite is a round-trip: it pushes events, queries them back through Loki's range API, and asserts the label set, the JSON log line and the timestamp survived. Without LOKI_ENDPOINT it skips itself with a visible label rather than passing silently.

To eyeball the result instead, point any Grafana at http://localhost:3100 and run {service="evlog-e2e"}.

Troubleshooting

Missing endpoint. LOKI_ENDPOINT is unset and no endpoint was passed. The drain logs [evlog/loki] Missing endpoint once and then does nothing, so requests are never blocked or failed.

Loki API error: 401. On Grafana Cloud, check that user is the numeric instance ID of the Loki datasource, not your account email.

Loki API error: 400 mentioning out-of-order entries. Older Loki versions reject entries older than the most recent one in a stream. evlog sorts within each push; if you still hit this, enable unordered_writes in Loki or reduce the batch interval.

Nothing appears in Grafana. Confirm the label set you are querying. Run {service=~".+"} first to see which streams arrived.

Direct API usage

Bypass the drain to push events yourself:

scripts/backfill.ts
import { sendBatchToLoki, sendToLoki } from 'evlog/loki'

await sendToLoki(event, { endpoint: 'http://localhost:3100' })
await sendBatchToLoki(events, { endpoint: 'http://localhost:3100' })

buildLokiPayload(), toLokiLabels(), and resolveLokiPushUrl() are also exported for custom transports.

Next steps