I'm excited to share that I recently wrote a blog post for the official OpenTelemetry site about monitoring LLMs, and I tried to include how you can use the existing O11y stack like Prometheus, jaeger and Grafana to have complete e2e monitoring for the LLMs apps using an OTel auto instrumentation library called OpenLIT.
I'm excited to share that I recently wrote a blog post for the official OpenTelemetry site about monitoring LLMs, and I tried to include how you can use the existing O11y stack like Prometheus, jaeger and Grafana to have complete e2e monitoring for the LLMs apps using an OTel auto instrumentation library called OpenLIT.
Check it out here: https://opentelemetry.io/blog/2024/llm-observability/
I’d love to discuss more on this and any insights you all might have regarding LLM Stack observability.