{"id":55442,"date":"2026-07-07T16:22:54","date_gmt":"2026-07-07T23:22:54","guid":{"rendered":"https:\/\/www.griddb.net\/?p=55442"},"modified":"2026-07-15T16:30:12","modified_gmt":"2026-07-15T23:30:12","slug":"storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka","status":"publish","type":"post","link":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/","title":{"rendered":"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka"},"content":{"rendered":"<p>\nOpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single monitoring vendor, it defines a common way to generate, collect, and export telemetry across three signal types: metrics, traces, and logs; commonly called the three pillars of observability. Metrics are numeric measurements sampled over time, such as CPU utilization or request counts. Traces describe the path of a single request as it moves through a system, broken into individual units of work called <em>spans<\/em>. Logs are timestamped event records emitted by the application.\n<\/p>\n<p>\nThe value of the three pillars comes from being able to correlate across them: spotting a latency spike in a trace, then jumping to the exact log lines emitted during that request. Realizing that benefit requires the signals to live somewhere you can query them together, which is where GridDB Cloud comes in. All three signals are, at their core, streams of timestamped events, making them a natural fit for a time-series database.\n<\/p>\n<p>\nWe have written extensively about pairing <a href=\"https:\/\/www.griddb.net\/en\/?s=kafka&#038;lang=en\">Kafka with GridDB<\/a>, because a data-ingestion pipeline like Kafka fits well with what GridDB is built for. Previously we used the GridDB Kafka Connector to push time-series data to GridDB Cloud over the Web API. With the release of GridDB Cloud v3.2, we can now connect to a cloud instance natively, without the Web API, which opens up a broader set of tools. In this article, we will use that native connection to land all three OpenTelemetry signals into GridDB Cloud through Kafka. We will collect host metrics from a local machine, instrument a small application to emit traces and logs, route everything through Kafka, flatten the nested OTLP payloads with a Go bridge, and sink each signal into its own GridDB Cloud <code>TIME_SERIES<\/code> container using the GridDB Kafka Connector. We will then write queries that analyze the data, per-operation latency profiles, error rates, metric summaries, and a cross-signal lookup that connects a failing trace to the logs it produced.\n<\/p>\n<p>\nGridDB&#8217;s design favors one container per series, <code>TIME_SERIES<\/code> containers are optimized per series, and a query scoped to a single container is faster than filtering one large mixed container. That principle shapes the whole pipeline: rather than dumping raw OTLP into one place, we explode it into one container per metric, plus one container each for spans and logs.\n<\/p>\n<p>\nAt a high level, this is what we will build:\n<\/p>\n<ol>\n<li>Set up the OpenTelemetry Collector to scrape host metrics and receive application traces and logs, pushing each signal to Kafka.<\/li>\n<li>Set up Kafka to receive and store the raw OTLP data.<\/li>\n<li>Run a Go &#8220;bridge&#8221; that reads the dense, nested OTLP JSON and explodes it into flat, one-row-per-event topics \u2014 one per metric, one for spans, one for logs.<\/li>\n<li>Use the GridDB Kafka Connector to sink those flattened topics into GridDB Cloud.<\/li>\n<\/ol>\n<div class=\"clipboard\">\n<pre><code class=\"language-plaintext\">host metrics \u2500\u2510\r\n                  \u251c\u2500\u25ba OTel Collector \u2500\u25ba Kafka  [otel-metrics] [otel-traces] [otel-logs]\r\nInstrumented app \u2500\u2518                                       \u2502\r\n                                                       Go bridge  (flatten nested OTLP)\r\n                                                          \u2502\r\n        Kafka  [metric_system_cpu_utilization, ...]  [otel_spans]  [otel_logs]\r\n                                                          \u2502\r\n                                              Kafka Connect (GridDB sink)\r\n                                                          \u2502\r\n                          GridDB Cloud \u2014 one TIME_SERIES container per signal<\/code><\/pre>\n<\/div>\n<p><a href=\"\/wp-content\/uploads\/2026\/07\/GridDBWebinar.png\"><img fetchpriority=\"high\" decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/07\/GridDBWebinar.png\" alt=\"\" width=\"1280\" height=\"720\" class=\"aligncenter size-full wp-image-55432\" srcset=\"\/wp-content\/uploads\/2026\/07\/GridDBWebinar.png 1280w, \/wp-content\/uploads\/2026\/07\/GridDBWebinar-300x169.png 300w, \/wp-content\/uploads\/2026\/07\/GridDBWebinar-1024x576.png 1024w, \/wp-content\/uploads\/2026\/07\/GridDBWebinar-768x432.png 768w, \/wp-content\/uploads\/2026\/07\/GridDBWebinar-150x85.png 150w, \/wp-content\/uploads\/2026\/07\/GridDBWebinar-600x338.png 600w\" sizes=\"(max-width: 1280px) 100vw, 1280px\" \/><\/a><\/p>\n<p>\nOne naming detail to note up front: the raw input topics use hyphens (<code>otel-metrics<\/code>, <code>otel-traces<\/code>, <code>otel-logs<\/code>) and the bridge&#8217;s flattened output topics use underscores (<code>metric_*<\/code>, <code>otel_spans<\/code>, <code>otel_logs<\/code>). The output names double as GridDB container names, and the hyphen\/underscore split lets the sink&#8217;s topic filter target only the flattened topics.\n<\/p>\n<h2 id=\"opentelemetry\">OpenTelemetry<\/h2>\n<p>\nInstall OpenTelemetry from <a href=\"https:\/\/opentelemetry.io\/\">opentelemetry.io<\/a>. In this case the Collector was installed on bare metal. Once installed, configure <code>config.yaml<\/code> to describe how the Collector should behave. The configuration below sets up all three pillars: the <code>hostmetrics<\/code> scraper produces metrics, while the <code>otlp<\/code> receiver accepts traces and logs from an instrumented application. Each pipeline exports to Kafka with OTLP-JSON encoding:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-yaml\">receivers:\r\n  otlp:\r\n    protocols:\r\n      grpc:\r\n        endpoint: 0.0.0.0:4317\r\n      http:\r\n        endpoint: 0.0.0.0:4318\r\n  hostmetrics:\r\n    collection_interval: 10s\r\n    scrapers:\r\n      cpu:\r\n        metrics:\r\n          system.cpu.utilization:\r\n            enabled: true\r\n      memory:\r\n        metrics:\r\n          system.memory.utilization:\r\n            enabled: true\r\n      load:\r\n      disk:\r\n\r\nprocessors:\r\n  resourcedetection:\r\n    detectors: [system]\r\n    system:\r\n      hostname_sources: [os]\r\n  batch:\r\n    timeout: 5s\r\n    send_batch_size: 100\r\n\r\nexporters:\r\n  kafka:\r\n    brokers:\r\n      - localhost:9092\r\n    metrics:\r\n      topic: otel-metrics\r\n      encoding: otlp_json\r\n    traces:\r\n      topic: otel-traces\r\n      encoding: otlp_json\r\n    logs:\r\n      topic: otel-logs\r\n      encoding: otlp_json\r\n  debug:\r\n    verbosity: basic\r\n\r\nservice:\r\n  pipelines:\r\n    metrics:\r\n      receivers: [otlp, hostmetrics]\r\n      processors: [resourcedetection, batch]\r\n      exporters: [kafka, debug]\r\n    traces:\r\n      receivers: [otlp]\r\n      processors: [resourcedetection, batch]\r\n      exporters: [kafka, debug]\r\n    logs:\r\n      receivers: [otlp]\r\n      processors: [resourcedetection, batch]\r\n      exporters: [kafka, debug]<\/code><\/pre>\n<\/div>\n<p>\nThe <code>hostmetrics<\/code> scraper is the metrics source, configured at the bottom of the file in the <code>metrics<\/code> pipeline; it exports to the <code>otel-metrics<\/code> topic. The <code>resourcedetection<\/code> processor attaches the host&#8217;s name to every signal as a resource attribute. Start the Collector by pointing it at the config:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ otelcol-contrib --config ~\/otel-griddb\/config.yaml<\/code><\/pre>\n<\/div>\n<h3 id=\"instrumenting-an-application-for-traces-and-logs\">Instrumenting an Application for Traces and Logs<\/h3>\n<p>\nHost metrics arrive on their own through the scraper, but in a real world setting, you&#8217;d want to be collecting traces and logs coming from the application you&#8217;re monitoring. To mimic this sort of real-world-use-case, we have set up a small Python worker that simulates processing jobs and instrument it with OpenTelemetry&#8217;s auto-instrumentation. Auto-instrumentation works well in this case because the application does not need an SDK wired in by hand; the <code>opentelemetry-instrument<\/code> launcher configures everything based on the environment variables including the providers, exporters, and the logging handler.\n<\/p>\n<p>\nA representative worker:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-python\">import logging\r\nimport random\r\nimport time\r\nimport uuid\r\n\r\nfrom opentelemetry import trace\r\n\r\nlogging.basicConfig(level=logging.INFO, format=&quot;%(asctime)s %(levelname)s %(message)s&quot;)\r\nlog = logging.getLogger(&quot;job-worker&quot;)\r\ntracer = trace.get_tracer(&quot;job-worker&quot;)\r\n\r\nJOB_TYPES = [&quot;transcode_video&quot;, &quot;resize_image&quot;, &quot;send_email&quot;, &quot;build_report&quot;]\r\n\r\n\r\ndef process_job(job_type: str, job_id: str) -&gt; None:\r\n    with tracer.start_as_current_span(job_type) as span:\r\n        span.set_attribute(&quot;job.id&quot;, job_id)\r\n        span.set_attribute(&quot;job.type&quot;, job_type)\r\n\r\n        duration = random.uniform(0.05, 1.5)\r\n        log.info(&quot;job %s (%s) started&quot;, job_id, job_type)\r\n        time.sleep(duration)\r\n\r\n        if random.random() &lt; 0.1:  # occasionally fail, so the data has errors to query\r\n            span.set_status(trace.Status(trace.StatusCode.ERROR, &quot;job failed&quot;))\r\n            log.error(&quot;job %s (%s) failed after %.2fs&quot;, job_id, job_type, duration)\r\n            return\r\n\r\n        log.info(&quot;job %s (%s) completed in %.2fs&quot;, job_id, job_type, duration)\r\n\r\n\r\ndef main() -&gt; None:\r\n    log.info(&quot;worker starting&quot;)\r\n    while True:\r\n        job_type = random.choice(JOB_TYPES)\r\n        process_job(job_type, uuid.uuid4().hex[:8])\r\n        time.sleep(random.uniform(0.2, 0.8))\r\n\r\n\r\nif __name__ == &quot;__main__&quot;:\r\n    main()<\/code><\/pre>\n<\/div>\n<p>\nTelemetry destinations are set through environment variables, which the launcher reads to decide where to send spans and logs:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ export OTEL_EXPORTER_OTLP_ENDPOINT=http:\/\/localhost:4317\r\n$ export OTEL_EXPORTER_OTLP_PROTOCOL=grpc\r\n$ export OTEL_SERVICE_NAME=job-worker\r\n$ export OTEL_TRACES_EXPORTER=otlp\r\n$ export OTEL_LOGS_EXPORTER=otlp\r\n$ export OTEL_METRICS_EXPORTER=none<\/code><\/pre>\n<\/div>\n<p>\nRun the worker under the launcher:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ opentelemetry-instrument python worker.py<\/code><\/pre>\n<\/div>\n<p>\nAs a tip, you can keep these exports in a file (for example <code>otel-env.sh<\/code>) and load them with <code>source otel-env.sh<\/code>. Pasting a multi-line block directly into a shell can silently corrupt the values if the newlines arrive as the literal characters <code>\\n<\/code> \u2014 each variable then absorbs the leftover <code>export<\/code> keyword, producing names such as <code>otlpnexport<\/code> that fail at startup with <code>Requested component 'otlpnexport' not found<\/code>. Sourcing a file avoids the issue.\n<\/p>\n<p>\nThe <code>OTEL_SERVICE_NAME<\/code> value is attached to every span and log as the <code>service.name<\/code> resource attribute, which the bridge promotes to a dedicated column.\n<\/p>\n<h2 id=\"kafka\">Kafka<\/h2>\n<p>\nThis setup uses the KRaft build of Kafka, which no longer requires a separate ZooKeeper process. Install Kafka and start it by pointing at the default config; installed via Homebrew, it starts like this:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ kafka-server-start $(brew --prefix)\/etc\/kafka\/kraft\/server.properties<\/code><\/pre>\n<\/div>\n<h3 id=\"kafka-plugins-and-the-griddb-sink\">Kafka Plugins and the GridDB Sink<\/h3>\n<p>\nWiring Kafka to GridDB Cloud requires the GridDB connector and its supporting <code>gridstore<\/code> JARs. Gather them into a directory of your choosing \u2014 here, a <code>griddb<\/code> subdirectory under a <code>kafka-plugins<\/code> folder in the home directory:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ls ~\/kafka-plugins\/griddb\r\n$ griddb-kafka-connector-0.6.jar        gridstore-conf-5.8.0.jar\r\n$ gridstore-5.8.0.jar                   gridstore-jdbc-5.8.0.jar\r\n$ gridstore-advanced-5.8.0.jar          gridstore-jdbc-call-logging-5.8.0.jar\r\n$ gridstore-call-logging-5.8.0.jar<\/code><\/pre>\n<\/div>\n<p>\nThese JARs are gathered from the GridDB Cloud v3.2 support page. You will also need to build the GridDB Kafka Connector from <a href=\"https:\/\/github.com\/Imisrael\/griddb-kafka-connect\">this fork<\/a>, which adds support for the <code>connection.route<\/code> and <code>database<\/code> configuration properties required for GridDB Cloud&#8217;s native connection.\n<\/p>\n<p>\n> <strong>Author note:<\/strong> Link the v3.2 support page (the <code>[TODO v3.2 BLOG]<\/code> reference from the original draft).\n<\/p>\n<h3 id=\"kafka-connect\">Kafka Connect<\/h3>\n<p>\nKafka Connect needs a <code>connect-standalone.properties<\/code> worker configuration. Because the bridge emits Kafka Connect envelopes that carry an explicit schema, the value converter must have schemas enabled:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-properties\"># connect-standalone.properties\r\n\r\nbootstrap.servers=localhost:9092\r\n\r\nkey.converter=org.apache.kafka.connect.json.JsonConverter\r\nvalue.converter=org.apache.kafka.connect.json.JsonConverter\r\nkey.converter.schemas.enable=false\r\nvalue.converter.schemas.enable=true\r\n\r\noffset.storage.file.filename=\/tmp\/connect.offsets\r\noffset.flush.interval.ms=10000\r\n\r\n# Directory above the griddb folder; Connect scans subdirectories for plugins\r\nplugin.path=\/Users\/israelimru\/kafka-plugins\r\n\r\nrest.port=8083<\/code><\/pre>\n<\/div>\n<p>\nThe sink configuration tells Connect how to push data to GridDB Cloud. A single sink handles all three signal types: every flattened topic \u2014 metrics, spans, and logs \u2014 carries a <code>datetime<\/code> field and maps to a <code>TIME_SERIES<\/code> container, so one topic filter and one timestamp transform cover them all. The <code>TimestampConverter<\/code> is essential: it coerces the integer epoch-millis <code>datetime<\/code> into a real <code>Timestamp<\/code>, which is what makes GridDB create a <code>TIME_SERIES<\/code> container keyed on time rather than a collection with a plain integer column.\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-properties\">name=griddb-otel-sink\r\nconnector.class=com.github.griddb.kafka.connect.GriddbSinkConnector\r\ntasks.max=1\r\n\r\ncluster.name=[yourClusterName]\r\nuser=[yourUser]\r\npassword=[yourPassword]\r\nmulticast=false\r\nnotification.provider.url=[yourNotificationProviderURL]\r\n\r\nconnection.route=PUBLIC\r\ndatabase=[yourDatabase]\r\n\r\ncontainer.type=TIME_SERIES\r\ntopics.regex=metric_.*|otel_.*\r\n\r\n# Coerce the int64 epoch-millis datetime into a Timestamp before mapping,\r\n# so datetime becomes the TIMESTAMP row key of each TIME_SERIES container.\r\ntransforms=TimestampConverter\r\ntransforms.TimestampConverter.type=org.apache.kafka.connect.transforms.TimestampConverter$Value\r\ntransforms.TimestampConverter.field=datetime\r\ntransforms.TimestampConverter.target.type=Timestamp<\/code><\/pre>\n<\/div>\n<p>\nThe <code>topics.regex=metric_.<em>|otel_.<\/em><\/code> pattern matches the underscore output topics (<code>metric_system_cpu_utilization<\/code>, <code>otel_spans<\/code>, <code>otel_logs<\/code>) while ignoring the hyphenated raw inputs. Start the Connect worker with both files:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ connect-standalone ~\/otel-griddb\/connect-standalone.properties ~\/otel-griddb\/griddb-otel-sink.properties<\/code><\/pre>\n<\/div>\n<p>\nThe <code>connect-standalone<\/code> command runs the Kafka Connect worker \u2014 the link between Kafka topics and external systems. It loads the GridDB sink plugin and starts the job defined in the sink properties, which consumes from the matching topics and writes each record as a row in GridDB Cloud.\n<\/p>\n<h2 id=\"the-go-bridge\">The Go Bridge<\/h2>\n<p>\nOTLP JSON is deeply nested. A metrics message wraps <code>resourceMetrics \u2192 scopeMetrics \u2192 metrics \u2192 dataPoints<\/code>; a traces message wraps <code>resourceSpans \u2192 scopeSpans \u2192 spans<\/code>; logs follow the same shape. GridDB stores flat rows, so the bridge reads each raw OTLP topic and explodes it into flat, one-row-per-event records, each wrapped in a Kafka Connect envelope carrying its schema. The main loop dispatches on the source topic:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-go\">switch rec.Topic {\r\ncase metricsTopic:                  \/\/ otel-metrics\r\n    produced, err = explode(rec.Value)\r\ncase tracesTopic:                   \/\/ otel-traces\r\n    produced, err = explodeTraces(rec.Value)\r\ncase logsTopic:                     \/\/ otel-logs\r\n    produced, err = explodeLogs(rec.Value)\r\n}<\/code><\/pre>\n<\/div>\n<h3 id=\"metrics\">Metrics<\/h3>\n<p>\nThe metrics handler walks down to each data point and emits one flat row per point, routing it to a per-metric topic (<code>metricToTopic<\/code> turns <code>system.cpu.utilization<\/code> into <code>metric_system_cpu_utilization<\/code>):\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-go\">func explode(payload []byte) ([]*kgo.Record, error) {\r\n\tvar o otlpMetrics\r\n\tif err := json.Unmarshal(payload, &amp;o); err != nil {\r\n\t\treturn nil, fmt.Errorf(&quot;unmarshal otlp: %w&quot;, err)\r\n\t}\r\n\r\n\tvar out []*kgo.Record\r\n\tfor _, rm := range o.ResourceMetrics {\r\n\t\thost := hostFromResource(rm.Resource)\r\n\t\tfor _, sm := range rm.ScopeMetrics {\r\n\t\t\tfor _, m := range sm.Metrics {\r\n\t\t\t\tvar series *dataSeries\r\n\t\t\t\tswitch {\r\n\t\t\t\tcase m.Gauge != nil:\r\n\t\t\t\t\tseries = m.Gauge\r\n\t\t\t\tcase m.Sum != nil:\r\n\t\t\t\t\tseries = m.Sum\r\n\t\t\t\tdefault:\r\n\t\t\t\t\tcontinue \/\/ histograms not supported\r\n\t\t\t\t}\r\n\r\n\t\t\t\ttopic := metricToTopic(m.Name)\r\n\t\t\t\tfor _, dp := range series.DataPoints {\r\n\t\t\t\t\tval, ok := dpValue(dp)\r\n\t\t\t\t\tif !ok {\r\n\t\t\t\t\t\tcontinue\r\n\t\t\t\t\t}\r\n\t\t\t\t\tnanos, err := strconv.ParseInt(dp.TimeUnixNano, 10, 64)\r\n\t\t\t\t\tif err != nil {\r\n\t\t\t\t\t\tcontinue\r\n\t\t\t\t\t}\r\n\t\t\t\t\tenvelope := connectEnvelope{\r\n\t\t\t\t\t\tSchema: flatSchema,\r\n\t\t\t\t\t\tPayload: flatRow{\r\n\t\t\t\t\t\t\tDatetime: nanos \/ 1_000_000,\r\n\t\t\t\t\t\t\tValue:    val,\r\n\t\t\t\t\t\t\tHost:     host,\r\n\t\t\t\t\t\t\tUnit:     m.Unit,\r\n\t\t\t\t\t\t\tAttrs:    attrsToString(dp.Attributes),\r\n\t\t\t\t\t\t},\r\n\t\t\t\t\t}\r\n\t\t\t\t\tb, _ := json.Marshal(envelope)\r\n\t\t\t\t\tout = append(out, &amp;kgo.Record{Topic: topic, Value: b})\r\n\t\t\t\t}\r\n\t\t\t}\r\n\t\t}\r\n\t}\r\n\treturn out, nil\r\n}<\/code><\/pre>\n<\/div>\n<h3 id=\"traces\">Traces<\/h3>\n<p>\nThe traces handler pulls <code>service.name<\/code> and <code>host.name<\/code> from the resource, iterates down to individual spans, converts nanosecond timestamps to milliseconds, and computes each span&#8217;s duration from its start and end times:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-go\">func explodeTraces(payload []byte) ([]*kgo.Record, error) {\r\n\tvar o otlpTraces\r\n\tif err := json.Unmarshal(payload, &amp;o); err != nil {\r\n\t\treturn nil, fmt.Errorf(&quot;unmarshal otlp traces: %w&quot;, err)\r\n\t}\r\n\r\n\tvar out []*kgo.Record\r\n\tfor _, rs := range o.ResourceSpans {\r\n\t\thost := resourceAttr(rs.Resource, &quot;host.name&quot;)\r\n\t\tservice := resourceAttr(rs.Resource, &quot;service.name&quot;)\r\n\t\tfor _, ss := range rs.ScopeSpans {\r\n\t\t\tfor _, s := range ss.Spans {\r\n\t\t\t\tstartNs, err1 := strconv.ParseInt(s.StartTimeUnixNano, 10, 64)\r\n\t\t\t\tendNs, err2 := strconv.ParseInt(s.EndTimeUnixNano, 10, 64)\r\n\t\t\t\tif err1 != nil || err2 != nil {\r\n\t\t\t\t\tcontinue\r\n\t\t\t\t}\r\n\t\t\t\tenvelope := spanEnvelope{\r\n\t\t\t\t\tSchema: spanSchema,\r\n\t\t\t\t\tPayload: spanRow{\r\n\t\t\t\t\t\tDatetime:     startNs \/ 1_000_000,\r\n\t\t\t\t\t\tDurationMs:   (endNs - startNs) \/ 1_000_000,\r\n\t\t\t\t\t\tTraceID:      s.TraceID,\r\n\t\t\t\t\t\tSpanID:       s.SpanID,\r\n\t\t\t\t\t\tParentSpanID: s.ParentSpanID,\r\n\t\t\t\t\t\tName:         s.Name,\r\n\t\t\t\t\t\tKind:         s.Kind,\r\n\t\t\t\t\t\tStatusCode:   s.Status.Code,\r\n\t\t\t\t\t\tServiceName:  service,\r\n\t\t\t\t\t\tHost:         host,\r\n\t\t\t\t\t\tAttrs:        attrsToString(s.Attributes),\r\n\t\t\t\t\t},\r\n\t\t\t\t}\r\n\t\t\t\tb, _ := json.Marshal(envelope)\r\n\t\t\t\tout = append(out, &amp;kgo.Record{Topic: spansContainer, Value: b})\r\n\t\t\t}\r\n\t\t}\r\n\t}\r\n\treturn out, nil\r\n}<\/code><\/pre>\n<\/div>\n<p>\nIts schema leaves <code>datetime<\/code> as <code>int64<\/code>; the sink&#8217;s <code>TimestampConverter<\/code> promotes it to a GridDB <code>TIMESTAMP<\/code> row key:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-go\">var spanSchema = connectSchema{\r\n\tType: &quot;struct&quot;,\r\n\tName: &quot;otel_span&quot;,\r\n\tFields: []connectField{\r\n\t\t{Type: &quot;int64&quot;, Optional: false, Field: &quot;datetime&quot;},\r\n\t\t{Type: &quot;int64&quot;, Optional: false, Field: &quot;duration_ms&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;trace_id&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;span_id&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;parent_span_id&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;name&quot;},\r\n\t\t{Type: &quot;int32&quot;, Optional: true, Field: &quot;kind&quot;},\r\n\t\t{Type: &quot;int32&quot;, Optional: true, Field: &quot;status_code&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;service_name&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;host&quot;},\r\n\t\t{Type: &quot;string&quot;, Optional: true, Field: &quot;attrs&quot;},\r\n\t},\r\n}<\/code><\/pre>\n<\/div>\n<h3 id=\"logs\">Logs<\/h3>\n<p>\nThe logs handler is structurally identical, walking <code>resourceLogs \u2192 scopeLogs \u2192 logRecords<\/code> and emitting one flat row per record to the <code>otel_logs<\/code> topic. Each row carries the record&#8217;s severity number and text, its body, the <code>service.name<\/code>\/<code>host.name<\/code> from the resource, and \u2014 critically \u2014 the <code>trace_id<\/code> and <code>span_id<\/code> that tie the log back to the span that produced it. That correlation key is the reason for landing logs and traces in the same database, and we use it in the queries below.\n<\/p>\n<p>\nBuild and run the bridge:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ go build -o bridge\r\n$ .\/bridge<\/code><\/pre>\n<\/div>\n<p>\nIt logs each input record it explodes:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-plaintext\">2026\/05\/15 13:31:36 topic=otel-metrics offset=150 exploded into 58 records\r\n2026\/05\/15 13:31:37 topic=otel-traces  offset=151 exploded into 12 records\r\n2026\/05\/15 13:31:37 topic=otel-logs    offset=149 exploded into 9 records<\/code><\/pre>\n<\/div>\n<h2 id=\"running-everything\">Running Everything<\/h2>\n<p>\nWith every piece in place, start the components in order:\n<\/p>\n<ol>\n<li>Kafka (KRaft)<\/li>\n<li>OTel Collector<\/li>\n<li>The instrumented worker (<code>opentelemetry-instrument python worker.py<\/code>)<\/li>\n<li>The Go bridge<\/li>\n<li>Kafka Connect (<code>connect-standalone<\/code>)<\/li>\n<\/ol>\n<p>\nOnce running, Kafka Connect reports writing rows to GridDB Cloud, with a container per metric plus the span and log containers:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-plaintext\">[griddb-otel-sink|task-0] Put 1 record to buffer of container metric_system_cpu_load_average_15m\r\n[griddb-otel-sink|task-0] Put 1 record to buffer of container metric_system_disk_io\r\n[griddb-otel-sink|task-0] Put 1 record to buffer of container otel_spans\r\n[griddb-otel-sink|task-0] Put 1 record to buffer of container otel_logs<\/code><\/pre>\n<\/div>\n<p>\nIn the GridDB Cloud console, the containers appear as <code>TIME_SERIES<\/code>. The <code>otel_spans<\/code> container holds one row per span \u2014 <code>datetime<\/code> (the span start) as the <code>TIMESTAMP<\/code> row key, plus <code>duration_ms<\/code>, the ID fields (<code>trace_id<\/code>, <code>span_id<\/code>, <code>parent_span_id<\/code>), the operation <code>name<\/code>, <code>kind<\/code>, <code>status_code<\/code>, <code>service_name<\/code>, <code>host<\/code>, and a flattened <code>attrs<\/code> string. The <code>otel_logs<\/code> container holds one row per log record, keyed on <code>datetime<\/code>, with <code>severity_number<\/code>, <code>severity_text<\/code>, <code>body<\/code>, <code>trace_id<\/code>, <code>span_id<\/code>, <code>service_name<\/code>, <code>host<\/code>, and <code>attrs<\/code>. Each <code>metric_*<\/code> container holds <code>datetime<\/code>, <code>value<\/code>, <code>host<\/code>, <code>unit<\/code>, and <code>attrs<\/code>.\n<\/p>\n<p>Here&#8217;s a brief look at what our data looks like in the GridDB Cloud Portal<\/p>\n<p><a href=\"\/wp-content\/uploads\/2026\/07\/image.png\"><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/07\/image.png\" alt=\"\" width=\"1183\" height=\"766\" class=\"aligncenter size-full wp-image-55435\" srcset=\"\/wp-content\/uploads\/2026\/07\/image.png 1183w, \/wp-content\/uploads\/2026\/07\/image-300x194.png 300w, \/wp-content\/uploads\/2026\/07\/image-1024x663.png 1024w, \/wp-content\/uploads\/2026\/07\/image-768x497.png 768w, \/wp-content\/uploads\/2026\/07\/image-600x389.png 600w\" sizes=\"(max-width: 1183px) 100vw, 1183px\" \/><\/a><\/p>\n<p><a href=\"\/wp-content\/uploads\/2026\/07\/image-1.png\"><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/07\/image-1.png\" alt=\"\" width=\"1208\" height=\"484\" class=\"aligncenter size-full wp-image-55433\" srcset=\"\/wp-content\/uploads\/2026\/07\/image-1.png 1208w, \/wp-content\/uploads\/2026\/07\/image-1-300x120.png 300w, \/wp-content\/uploads\/2026\/07\/image-1-1024x410.png 1024w, \/wp-content\/uploads\/2026\/07\/image-1-768x308.png 768w, \/wp-content\/uploads\/2026\/07\/image-1-600x240.png 600w\" sizes=\"(max-width: 1208px) 100vw, 1208px\" \/><\/a><\/p>\n<p><a href=\"\/wp-content\/uploads\/2026\/07\/image-2.png\"><img loading=\"lazy\" decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/07\/image-2.png\" alt=\"\" width=\"1232\" height=\"742\" class=\"aligncenter size-full wp-image-55434\" srcset=\"\/wp-content\/uploads\/2026\/07\/image-2.png 1232w, \/wp-content\/uploads\/2026\/07\/image-2-300x181.png 300w, \/wp-content\/uploads\/2026\/07\/image-2-1024x617.png 1024w, \/wp-content\/uploads\/2026\/07\/image-2-768x463.png 768w, \/wp-content\/uploads\/2026\/07\/image-2-600x361.png 600w\" sizes=\"(max-width: 1232px) 100vw, 1232px\" \/><\/a><\/p>\n<h2 id=\"querying-the-data\">Querying the Data<\/h2>\n<p>\nStoring telemetry is only useful if you can ask questions of it. The following queries move from simple feeds to genuine analysis, one pillar at a time, and finish by crossing between them.\n<\/p>\n<h3 id=\"metrics\">Metrics<\/h3>\n<p>\nFor a single metric, a one-line aggregate summarizes its range over the whole collection period \u2014 useful for spotting how much disk io has run:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT MIN(value) AS min_util,\r\n       AVG(value) AS avg_util,\r\n       MAX(value) AS max_util\r\nFROM metric_system_disk_io;<\/code><\/pre>\n<\/div>\n<p>\nExample run using the GridDB Cloud CLI Tool:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT MIN(value) AS min_util, AVG(value) AS avg_util, MAX(value) AS max_util FROM metric_system_disk_io;&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT MIN(value) AS min_util, AVG(value) AS avg_util, MAX(value) AS max_util FROM metric_system_disk_io;&quot; }]\r\n$ min_util,avg_util,max_util,\r\n$ [1.00627853312e+11 1.2719715549835805e+11 3.24776685568e+11]<\/code><\/pre>\n<\/div>\n<p>\nThe latest readings give a live view of a single series:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT datetime, value, host\r\nFROM metric_system_disk_io\r\nORDER BY datetime DESC\r\nLIMIT 20;<\/code><\/pre>\n<\/div>\n<p>\nExample run:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT datetime, value, host FROM metric_system_disk_io ORDER BY datetime DESC LIMIT 20;&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT datetime, value, host FROM metric_system_disk_io ORDER BY datetime DESC LIMIT 20;&quot; }]\r\n$ datetime,value,host,\r\n$ [2026-05-27T20:54:31.978Z 1.65120098304e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:54:21.979Z 1.65116936192e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:54:11.979Z 1.6481210368e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:54:01.980Z 1.64800229376e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:51.979Z 1.64774305792e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:41.979Z 1.64678656e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:31.979Z 1.64668919808e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:21.980Z 1.6465928192e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:11.979Z 1.64600336384e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:53:01.979Z 1.64591157248e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:51.996Z 1.64583391232e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:41.980Z 1.64520382464e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:31.979Z 1.64511981568e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:21.980Z 1.64502102016e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:11.979Z 1.64488118272e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:52:01.979Z 1.64480098304e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:51:51.979Z 1.6446855168e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:51:41.979Z 1.6444940288e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:51:31.979Z 1.64428091392e+11 Israels-Mac.local]\r\n$ [2026-05-27T20:51:21.979Z 1.64419739648e+11 Israels-Mac.local]<\/code><\/pre>\n<\/div>\n<h3 id=\"traces\">Traces<\/h3>\n<p>\nA few conventions matter for the trace and log queries. A span&#8217;s <code>status_code<\/code> is <code>0<\/code> for UNSET, <code>1<\/code> for OK, and <code>2<\/code> for ERROR. A log record&#8217;s <code>severity_number<\/code> follows fixed ranges: 1\u20134 TRACE, 5\u20138 DEBUG, 9\u201312 INFO, 13\u201316 WARN, 17\u201320 ERROR, 21\u201324 FATAL.\n<\/p>\n<p>\nThe first real question for any traced system is where the time is going. Because each span carries its own <code>duration_ms<\/code>, a single grouped query produces a full latency profile per operation \u2014 minimum, average, and maximum duration with the call count:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT name,\r\n       COUNT(*)         AS calls,\r\n       MIN(duration_ms) AS min_ms,\r\n       AVG(duration_ms) AS avg_ms,\r\n       MAX(duration_ms) AS max_ms\r\nFROM otel_spans\r\nGROUP BY name\r\nORDER BY avg_ms DESC;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT name, COUNT(*) AS calls, MIN(duration_ms) AS min_ms, AVG(duration_ms) AS avg_ms, MAX(duration_ms) AS max_ms FROM otel_spans GROUP BY name ORDER BY avg_ms DESC&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT name, COUNT(*) AS calls, MIN(duration_ms) AS min_ms, AVG(duration_ms) AS avg_ms, MAX(duration_ms) AS max_ms FROM otel_spans GROUP BY name ORDER BY avg_ms DESC&quot; }]\r\n$ name,calls,min_ms,avg_ms,max_ms,\r\n$ [job.transcode_video 3535 11 1455.5188118811882 953212]\r\n$ [job.generate_report 3609 12 889.7420338043779 1.000412e+06]\r\n$ [job.sync_inventory 3569 24 285.75231157186886 2580]\r\n$ [job.resize_image 3587 7 120.47058823529412 1619]\r\n$ [job.send_email 3660 15 57.44672131147541 207]<\/code><\/pre>\n<\/div>\n<p>\nA high <code>max_ms<\/code> against a low <code>avg_ms<\/code> points to intermittent outliers. To inspect those outliers directly, sort by duration:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT ts, name, duration_ms, trace_id\r\nFROM otel_spans\r\nORDER BY duration_ms DESC\r\nLIMIT 10;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT ts, name, duration_ms, trace_id FROM otel_spans ORDER BY duration_ms DESC LIMIT 10&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT ts, name, duration_ms, trace_id FROM otel_spans ORDER BY duration_ms DESC LIMIT 10&quot; }]\r\n$ ts,name,duration_ms,trace_id,\r\n$ [2026-07-01T21:39:34.774Z job.generate_report 1.000412e+06 11bedd5adde787c507fe6700220f2719]\r\n$ [2026-07-01T21:18:32.090Z job.transcode_video 953212 4c945d48dab3de71cf43e2be37bcc845]\r\n$ [2026-07-01T21:16:27.534Z job.transcode_video 107162 b23966ac1330eaff79a17b0c0adeb9b4]\r\n$ [2026-07-01T19:16:53.336Z job.transcode_video 8010 501f49c5d7a8b5e8774c2e8f013f27bf]\r\n$ [2026-07-01T22:26:21.396Z job.transcode_video 8010 9064d16e9f05ef6e67d2b7f32bc4368a]\r\n$ [2026-07-01T21:04:59.306Z job.transcode_video 8007 1a96c129efbba1ab93fc5263242b9f52]\r\n$ [2026-07-01T20:22:30.204Z job.transcode_video 8007 ad9e78a3ab6f3ab1bc21c9454395f6c5]\r\n$ [2026-07-01T20:44:08.697Z job.transcode_video 8006 b7429f02f990a3e7decd314ec8a61e45]\r\n$ [2026-07-01T20:29:03.554Z job.transcode_video 8006 84f1aa7a770aa5fc29ca409e71b93b6a]\r\n$ [2026-07-01T20:03:24.204Z job.transcode_video 8006 57207e6dedc7c6e24e76cb061e8e7022]<\/code><\/pre>\n<\/div>\n<p>\nReliability is the next question. Since <code>status_code = 2<\/code> marks an errored span, error counts per operation come from a single filtered aggregate:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT name, COUNT(*) AS errors\r\nFROM otel_spans\r\nWHERE status_code = 2\r\nGROUP BY name\r\nORDER BY errors DESC;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT name, COUNT(*) AS errors FROM otel_spans WHERE status_code = 2 GROUP BY name ORDER BY errors DESC&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT name, COUNT(*) AS errors FROM otel_spans WHERE status_code = 2 GROUP BY name ORDER BY errors DESC&quot; }]\r\n$ name,errors,\r\n$ [job.generate_report 344]\r\n$ [job.transcode_video 208]\r\n$ [job.sync_inventory 142]\r\n$ [job.resize_image 69]\r\n$ [job.send_email 34]<\/code><\/pre>\n<\/div>\n<h3 id=\"logs\">Logs<\/h3>\n<p>\nA severity breakdown gives a quick health summary:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT severity_text, COUNT(*) AS count\r\nFROM otel_logs\r\nGROUP BY severity_text\r\nORDER BY count DESC;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT severity_text, COUNT(*) AS count FROM otel_logs GROUP BY severity_text ORDER BY count DESC&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT severity_text, COUNT(*) AS count FROM otel_logs GROUP BY severity_text ORDER BY count DESC&quot; }]\r\n$ severity_text,count,\r\n$ [INFO 17174]\r\n$ [ERROR 797]<\/code><\/pre>\n<\/div>\n<p>\nFiltering on the severity number isolates everything at ERROR level or above:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT datetime, severity_text, service_name, body\r\nFROM otel_logs\r\nWHERE severity_number &gt;= 17\r\nORDER BY datetime DESC\r\nLIMIT 20;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT ts, severity_text, service_name, body FROM otel_logs WHERE severity_number &gt;= 17 ORDER BY ts DESC LIMIT 20&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT ts, severity_text, service_name, body FROM otel_logs WHERE severity_number &gt;= 17 ORDER BY ts DESC LIMIT 20&quot; }]\r\n$ ts,severity_text,service_name,body,\r\n$ [2026-07-01T22:48:29.403Z ERROR job-worker job d9dd3342 (generate_report) failed after 0.47s: rate limited]\r\n$ [2026-07-01T22:48:18.950Z ERROR job-worker job 070712fb (sync_inventory) failed after 0.06s: invalid payload]\r\n$ [2026-07-01T22:48:13.213Z ERROR job-worker job d56f92f3 (generate_report) failed after 0.39s: downstream timeout]\r\n$ [2026-07-01T22:48:11.816Z ERROR job-worker job 05ef9096 (generate_report) failed after 0.25s: invalid payload]\r\n$ [2026-07-01T22:48:00.613Z ERROR job-worker job 4441a1ec (transcode_video) failed after 3.86s: downstream timeout]\r\n$ [2026-07-01T22:47:55.657Z ERROR job-worker job 48491076 (generate_report) failed after 0.64s: connection refused]\r\n$ [2026-07-01T22:47:28.637Z ERROR job-worker job 51316b2b (transcode_video) failed after 0.17s: rate limited]\r\n$ [2026-07-01T22:47:26.343Z ERROR job-worker job b554b041 (transcode_video) failed after 0.53s: connection refused]\r\n$ [2026-07-01T22:47:24.968Z ERROR job-worker job 3e71df8a (generate_report) failed after 0.67s: downstream timeout]\r\n$ [2026-07-01T22:47:14.096Z ERROR job-worker job 31e1be77 (transcode_video) failed after 0.05s: invalid payload]\r\n$ [2026-07-01T22:47:12.386Z ERROR job-worker job f638a350 (transcode_video) failed after 0.75s: connection refused]\r\n$ [2026-07-01T22:47:01.852Z ERROR job-worker job 3fe1ac79 (generate_report) failed after 0.45s: invalid payload]\r\n$ [2026-07-01T22:46:36.300Z ERROR job-worker job 346819a2 (generate_report) failed after 1.38s: downstream timeout]\r\n$ [2026-07-01T22:46:28.549Z ERROR job-worker job 483cabb8 (resize_image) failed after 0.05s: connection refused]\r\n$ [2026-07-01T22:46:14.511Z ERROR job-worker job c438270a (transcode_video) failed after 0.09s: connection refused]\r\n$ [2026-07-01T22:45:49.406Z ERROR job-worker job 60e89b69 (generate_report) failed after 0.46s: connection refused]\r\n$ [2026-07-01T22:45:14.076Z ERROR job-worker job e3404a92 (send_email) failed after 0.09s: rate limited]\r\n$ [2026-07-01T22:45:11.384Z ERROR job-worker job e5eaabd8 (generate_report) failed after 1.36s: invalid payload]\r\n$ [2026-07-01T22:45:08.027Z ERROR job-worker job e207ee04 (generate_report) failed after 1.64s: invalid payload]\r\n$ [2026-07-01T22:44:55.155Z ERROR job-worker job 3af41a8b (generate_report) failed after 1.43s: invalid payload]<\/code><\/pre>\n<\/div>\n<h3 id=\"crossing-signals\">Crossing Signals<\/h3>\n<p>\nThe most compelling result of collectingg traces and logs in the same database is correlation. Suppose the error query above flags an operation that fails often. We can take its most recent failing trace and pull every log record emitted during that exact request, joined on <code>trace_id<\/code>. First identify the trace:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT trace_id, name, duration_ms\r\nFROM otel_spans\r\nWHERE status_code = 2\r\nORDER BY datetime DESC\r\nLIMIT 1;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT trace_id, name, duration_ms FROM otel_spans WHERE status_code = 2 ORDER BY ts DESC LIMIT 1&quot; -r \r\n$ [{&quot;stmt&quot;: &quot;SELECT trace_id, name, duration_ms FROM otel_spans WHERE status_code = 2 ORDER BY ts DESC LIMIT 1&quot; }]\r\n$ trace_id,name,duration_ms,\r\n$ [693c5d8f3479288991b91dafd43d20f1 job.generate_report 473]<\/code><\/pre>\n<\/div>\n<p>\nThen retrieve its logs in order:\n<\/p>\n<div class=\"clipboard\">\n<pre><code class=\"language-sql\">SELECT datetime, severity_text, body, span_id\r\nFROM otel_logs\r\nWHERE trace_id = &#x27;&lt;trace_id from the previous result&gt;&#x27;\r\nORDER BY datetime;<\/code><\/pre>\n<\/div>\n<div class=\"clipboard\">\n<pre><code class=\"language-sh\">$ \u279c  ~ griddb-cloud-cli sql query -s &quot;SELECT ts, severity_text, body, span_id FROM otel_logs WHERE trace_id = &#x27;693c5d8f3479288991b91dafd43d20f1&#x27; ORDER BY ts&quot; -r\r\n$ [{&quot;stmt&quot;: &quot;SELECT ts, severity_text, body, span_id FROM otel_logs WHERE trace_id = &#x27;693c5d8f3479288991b91dafd43d20f1&#x27; ORDER BY ts&quot; }]\r\n$ ts,severity_text,body,span_id,\r\n$ [2026-07-01T22:48:29.403Z ERROR job d9dd3342 (generate_report) failed after 0.47s: rate limited 7492c1f55f487fe2]<\/code><\/pre>\n<\/div>\n<p>\nIn two short queries we go from &#8220;this operation is failing&#8221; to the precise log lines explaining why  an investigation that typically spans two separate systems, performed here against one database.\n<\/p>\n<h2 id=\"conclusion\">Conclusion<\/h2>\n<p>\nWith this pipeline in place, all three OpenTelemetry signals land in GridDB Cloud through a single path: the Collector handles ingestion and fan-out to Kafka, the Go bridge flattens nested OTLP into one clean row per event, and the GridDB Kafka Connector \u2014 using GridDB Cloud v3.2&#8217;s native connection \u2014 sinks each signal into a purpose-built <code>TIME_SERIES<\/code> container. Because the data is queryable with standard aggregates and the trace identifier is shared between spans and logs, GridDB becomes a single backend for the correlated analysis that observability work depends on: metric summaries, latency profiling, error attribution, and trace-to-log drilldown.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single monitoring vendor, it defines a common way to generate, collect, and export telemetry across three signal types: metrics, traces, and logs; commonly called the three pillars of observability. Metrics are numeric measurements sampled over time, such [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":55444,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[121],"tags":[],"class_list":["post-55442","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.1.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT<\/title>\n<meta name=\"description\" content=\"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT\" \/>\n<meta property=\"og:description\" content=\"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\" \/>\n<meta property=\"og:site_name\" content=\"GridDB: Open Source Time Series Database for IoT\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/griddbcommunity\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-07T23:22:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-07-15T23:30:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.griddb.net\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-1024x376.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"376\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Israel\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@GridDBCommunity\" \/>\n<meta name=\"twitter:site\" content=\"@GridDBCommunity\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Israel\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\"},\"author\":{\"name\":\"Israel\",\"@id\":\"https:\/\/www.griddb.net\/en\/#\/schema\/person\/c8a430e7156a9e10af73b1fbb46c2740\"},\"headline\":\"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka\",\"datePublished\":\"2026-07-07T23:22:54+00:00\",\"dateModified\":\"2026-07-15T23:30:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\"},\"wordCount\":1853,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.griddb.net\/en\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage\"},\"thumbnailUrl\":\"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png\",\"articleSection\":[\"Blog\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\",\"url\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\",\"name\":\"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT\",\"isPartOf\":{\"@id\":\"https:\/\/www.griddb.net\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage\"},\"thumbnailUrl\":\"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png\",\"datePublished\":\"2026-07-07T23:22:54+00:00\",\"dateModified\":\"2026-07-15T23:30:12+00:00\",\"description\":\"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage\",\"url\":\"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png\",\"contentUrl\":\"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png\",\"width\":2560,\"height\":940},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.griddb.net\/en\/#website\",\"url\":\"https:\/\/www.griddb.net\/en\/\",\"name\":\"GridDB: Open Source Time Series Database for IoT\",\"description\":\"GridDB is an open source time-series database with the performance of NoSQL and convenience of SQL\",\"publisher\":{\"@id\":\"https:\/\/www.griddb.net\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.griddb.net\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.griddb.net\/en\/#organization\",\"name\":\"Fixstars\",\"url\":\"https:\/\/www.griddb.net\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.griddb.net\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/griddb.net\/wp-content\/uploads\/2019\/04\/fixstars_logo_web_tagline.png\",\"contentUrl\":\"https:\/\/griddb.net\/wp-content\/uploads\/2019\/04\/fixstars_logo_web_tagline.png\",\"width\":200,\"height\":83,\"caption\":\"Fixstars\"},\"image\":{\"@id\":\"https:\/\/www.griddb.net\/en\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/griddbcommunity\/\",\"https:\/\/x.com\/GridDBCommunity\",\"https:\/\/www.linkedin.com\/company\/griddb-by-toshiba\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.griddb.net\/en\/#\/schema\/person\/c8a430e7156a9e10af73b1fbb46c2740\",\"name\":\"Israel\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.griddb.net\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4df8cfc155402a2928d11f80b0220037b8bd26c4f1b19c4598d826e0306e6307?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4df8cfc155402a2928d11f80b0220037b8bd26c4f1b19c4598d826e0306e6307?s=96&d=mm&r=g\",\"caption\":\"Israel\"},\"url\":\"https:\/\/www.griddb.net\/en\/author\/israel\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT","description":"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/","og_locale":"en_US","og_type":"article","og_title":"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT","og_description":"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single","og_url":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/","og_site_name":"GridDB: Open Source Time Series Database for IoT","article_publisher":"https:\/\/www.facebook.com\/griddbcommunity\/","article_published_time":"2026-07-07T23:22:54+00:00","article_modified_time":"2026-07-15T23:30:12+00:00","og_image":[{"width":1024,"height":376,"url":"https:\/\/www.griddb.net\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-1024x376.png","type":"image\/png"}],"author":"Israel","twitter_card":"summary_large_image","twitter_creator":"@GridDBCommunity","twitter_site":"@GridDBCommunity","twitter_misc":{"Written by":"Israel","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#article","isPartOf":{"@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/"},"author":{"name":"Israel","@id":"https:\/\/www.griddb.net\/en\/#\/schema\/person\/c8a430e7156a9e10af73b1fbb46c2740"},"headline":"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka","datePublished":"2026-07-07T23:22:54+00:00","dateModified":"2026-07-15T23:30:12+00:00","mainEntityOfPage":{"@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/"},"wordCount":1853,"commentCount":0,"publisher":{"@id":"https:\/\/www.griddb.net\/en\/#organization"},"image":{"@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage"},"thumbnailUrl":"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png","articleSection":["Blog"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/","url":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/","name":"Storing OpenTelemetry Metrics, Traces, and Logs in GridDB Cloud with Kafka | GridDB: Open Source Time Series Database for IoT","isPartOf":{"@id":"https:\/\/www.griddb.net\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage"},"image":{"@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage"},"thumbnailUrl":"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png","datePublished":"2026-07-07T23:22:54+00:00","dateModified":"2026-07-15T23:30:12+00:00","description":"OpenTelemetry (OTel) is an open-source, vendor-neutral observability framework for cloud-native software. Rather than tying an application to a single","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.griddb.net\/en\/blog\/storing-opentelemetry-metrics-traces-and-logs-in-griddb-cloud-with-kafka\/#primaryimage","url":"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png","contentUrl":"\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_fka5ntfka5ntfka5-scaled.png","width":2560,"height":940},{"@type":"WebSite","@id":"https:\/\/www.griddb.net\/en\/#website","url":"https:\/\/www.griddb.net\/en\/","name":"GridDB: Open Source Time Series Database for IoT","description":"GridDB is an open source time-series database with the performance of NoSQL and convenience of SQL","publisher":{"@id":"https:\/\/www.griddb.net\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.griddb.net\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.griddb.net\/en\/#organization","name":"Fixstars","url":"https:\/\/www.griddb.net\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.griddb.net\/en\/#\/schema\/logo\/image\/","url":"https:\/\/griddb.net\/wp-content\/uploads\/2019\/04\/fixstars_logo_web_tagline.png","contentUrl":"https:\/\/griddb.net\/wp-content\/uploads\/2019\/04\/fixstars_logo_web_tagline.png","width":200,"height":83,"caption":"Fixstars"},"image":{"@id":"https:\/\/www.griddb.net\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/griddbcommunity\/","https:\/\/x.com\/GridDBCommunity","https:\/\/www.linkedin.com\/company\/griddb-by-toshiba"]},{"@type":"Person","@id":"https:\/\/www.griddb.net\/en\/#\/schema\/person\/c8a430e7156a9e10af73b1fbb46c2740","name":"Israel","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.griddb.net\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/4df8cfc155402a2928d11f80b0220037b8bd26c4f1b19c4598d826e0306e6307?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4df8cfc155402a2928d11f80b0220037b8bd26c4f1b19c4598d826e0306e6307?s=96&d=mm&r=g","caption":"Israel"},"url":"https:\/\/www.griddb.net\/en\/author\/israel\/"}]}},"_links":{"self":[{"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/posts\/55442","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/comments?post=55442"}],"version-history":[{"count":2,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/posts\/55442\/revisions"}],"predecessor-version":[{"id":55445,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/posts\/55442\/revisions\/55445"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/media\/55444"}],"wp:attachment":[{"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/media?parent=55442"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/categories?post=55442"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.griddb.net\/en\/wp-json\/wp\/v2\/tags?post=55442"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}