OpenLIT

To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM application to HyperDX, follow the below steps:

This Guide Integrates:  Metrics  ·  Traces

Getting Started

Install OpenTelemetry Collector (Optional)

This step is optional if you have the OpenTelemetry Collector already running.

For detailed installation instructions for the OpenTelemetry Collector, please refer to the OpenTelemetry Collector Documentation (opens in a new tab). It provides comprehensive steps to get you up and running with the Collector on various platforms.

Configure the OpenTelemetry Collector

1. Configure HTTP Receiver: In the receivers section of your OpenTelemetry Collector config, ensure the http receiver is set with endpoint: 0.0.0.0:4318.

   receivers:
     otlp:
       protocols:
         http:
           endpoint: 0.0.0.0:4318

2. Define Exporters: Add otlp exporter to export metrics and traces to HyperDX.

   exporters:
     # HTTP setup
     otlphttp/hdx:
       endpoint: 'https://in-otel.hyperdx.io'
       headers:
       authorization: YOUR_HYPERDX_API_KEY_HERE
       compression: gzip
    
     # gRPC setup (alternative)
     otlp/hdx:
       endpoint: 'in-otel.hyperdx.io:4317'
       headers:
       authorization: YOUR_HYPERDX_API_KEY_HERE
       compression: gzip

   Replace:

   1. YOUR_HYPERDX_API_KEY_HERE with the your HyperDX API Key.
      • Example - x6xx7265-43x3-476x-1112-x9x52x29xxxx

3. Assign Exporters to Pipelines: Link otlphttp/hdx to service.pipelines.traces and service.pipelines.metrics for data export.

   service:
     pipelines:
       traces:
         receivers: [ otlp ]
         exporters: [ otlphttp/hdx ]
       metrics:
         receivers: [ otlp ]
         exporters: [ otlphttp/hdx ]

Complete Configuration Example

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
 
processors:
  batch:
  memory_limiter:
    # 80% of maximum memory up to 2G
    limit_mib: 1500
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s
 
exporters:
  otlphttp/hdx:
    endpoint: 'https://in-otel.hyperdx.io'
    headers:
    authorization: YOUR_HYPERDX_API_KEY_HERE
    compression: gzip
 
service:
  pipelines:
    traces:
      receivers: [ otlp ]
      processors: [ memory_limiter, batch ]
      exporters: [ otlphttp/hdx ]
    metrics:
      receivers: [ otlp ]
      processors: [ memory_limiter, batch ]
      exporters: [ otlphttp/hdx ]

Configuring OpenLIT SDK

Add the following two lines to your application code:

import openlit
 
openlit.init(
  otlp_endpoint="YOUR_OTELCOL_URL:4318", 
)

Replace:

1. YOUR_OTELCOL_URL:4318 with the URL HTTP endpoint of your OpenTelemetry Collector.
   • Example - http://127.0.0.1:4318

or else you can also provide the OpenTelemetry collector URL using the OTEL_EXPORTER_OTLP_ENDPOINT varibale

import openlit
 
openlit.init()

Run the following command to configure the OTEL endpoint and headers to send metrics and traces to Grafana Cloud:

export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_OTELCOL_URL:4318"

Replace:

1. YOUR_OTELCOL_URL:4318 with the URL HTTP endpoint of your OpenTelemetry Collector.

• Example - http://127.0.0.1:4318

Refer to the OpenLIT Python SDK repository (opens in a new tab) for more advanced configurations and use cases.

Start monitoring using the Pre-built HyperDX Dashboard

You can use the pre-built HyperDX dashboard (opens in a new tab) to monitor your LLM application's metrics and traces. Make sure to click "Save Dashboard" if you want to save the dashboard in your account for future use.