Post

Langfuse: Why It Matters and How to Get Started

Langfuse: Why It Matters and How to Get Started
📑 Table of Contents

What is Langfuse?

Langfuse is an open-source observability platform designed specifically for LLM-based applications.At its core, it is: A backend + dashboard for LLM observability.Think of it like Jaeger, Grafana, or DataDog, but tailored for LLMs. It provides deep tracing and analytics for: Prompts & responses,Nested function/tool calls,Latency & token usage,API costs,Scoring, metadata, and more.So that you can understand, debug, and improve your LLM-powered workflows with full visibility.

NOTE: Langfuse is not limited to instrumenting only LLM flows. It is fundamentally an instrumentation tool. It just happens to be tailored for LLM flows and accommodates its naming and concepts accordingly.

Langfuse has two components

  • 🖥️ Dashboard (Backend + UI)

    • Focuses on LLM-specific data: token usage, prompts, responses, retries, tool-calling, costs, etc.
    • Powered by ClickHouse (for fast analytics), Redis, and S3 (for large trace payloads).
  • 🧰 Langfuse SDK

    • This is what you integrate into your app (available in Python, JS/TS, etc).
    • It wraps the OTel SDK and:
      • Provides simplified classes and APIs to create and track LLM spans (e.g. Trace, Span, Score)
      • Sends this data to the Langfuse backend using standard OTel transport mechanisms

It’s basically a developer-friendly layer over OpenTelemetry with LLM semantics.

Langfuse = [Jaeger for LLMs] + [OTel wrapper SDK] + [LLM-specific dashboarding & tooling]

So, What’s the idea behind creating Langfuse?

Langfuse is built primarily for telemetry in LLM- and AI-agent-driven applications.But here’s the nuance — while you can use it to record any kind of data, the UI and backend are so focused on LLM concepts, it feels like it was made exclusively for AI workflows.


Let’s start simple you make an LLM call — prompt in, response out. Maybe you log token usage, model, cost. All good.

But then reality kicks in: Outputs vary across calls , One call can trigger others — tools, APIs, DB queries , It’s all part of a bigger flow: a session, task, or conversation

Think about it — An application starts with a user question. It calls an LLM, maybe queries a database, then calls another LLM again, possibly invoking a tool in between… and so on. Eventually, the whole thing concludes.But this entire journey — from that initial user input to all the hops in between —

User Question or Prompts → LLM #1 → DB Query → LLM #2 → Tool Call → Final Response also Same LLm and prompt can yield different responses as well

How do you trace that?

First Thought: Let’s Build an Instrumentation Layer

You figure: let’s create a system that lets developers define and track these flows — like manual logging, but structured and traceable. Every step in the flow is recorded and sent to a dashboard where you can replay the journey, understand the behavior, and debug when things go wrong.

Sounds great, right?

🤔 But wait a second… isn’t there already something that does this? OpenTelemetry (OTEL)? it’s battle-tested and robust

Yes OTEL already gives you:

  • ✅ A solid trace data model
  • ✅ Standardized spans and context propagation
  • ✅ Multiple language SDKs

why not just stand on OTEL’s shoulders?


Things like: prompt, completion, token_usage, model, latency, cost, score, tool_calls

Here’s the Clever Move:

  1. Use OTEL’s trace model as the foundation : (It’s already great for tracking distributed systems and flow lifecycles.)

  2. Extend it with LLM-specific metadata
    • Things like: prompt, completion, token_usage, model, latency, cost, score, tool_calls
    • All of this fits neatly into the attributes section of the trace datamodel (a Span) which is used to transfer data — no need to break the OTEL model.Examples in the below section
      1
      2
      3
      4
      5
      6
      7
      8
      9
      
       "attributes": {
           "langfuse.observation.input": "{\"sql\": \"SELECT * FROM users WHERE active=1\"}",
           "langfuse.observation.output": "{\"rows\": 42}",
           "langfuse.observation.metadata.db": "users",
           "langfuse.observation.type": "span",
           "langfuse.observation.level": "INFO",
           "langfuse.observation.status_message": "Fetched 42 active users",
           "langfuse.observation.metadata.my_new_metric": "Somevalue"
       }
      
  3. Build a purpose-built dashboard and backend
    • One that doesn’t just show spans and traces like Jaeger or Grafana, but instead understands LLM-specific attributes and visualizes them meaningfully.

💥 Boom — that’s Langfuse.

It’s a specialized telemetry platform, built on the solid foundation of OpenTelemetry, but enhanced with a deep understanding of LLM and AI application needs.

🤖 why dont we use OTel ,What’s Missing for LLMs/Agents?

If you just used OTel raw, you’d get:

  • ✅ A trace → good, but it doesn’t know what a “prompt” is.
  • ✅ A span → good, but it doesn’t know how to display input/output clearly.
  • ✅ A metric → good, but you’d have to invent something like tokens_used as a metric yourself.
  • ✅ A log → good, but where do you put something like model=GPT-4 neatly?

👉 You could still do it, but it would be like forcing LLMs into a generic observability box.
You’d have to:

  • 🛠️ Define custom conventions the schemantics (naming traces, spans, tags)
  • 🖥️ Build your own dashboard on top of Grafana/Jaeger
  • 🔗 Manually correlate prompts/responses with metrics

That’s heavy work.

How langfuse solves it

🔁 Reuse OTel’s Primitives

  • Trace ↔ Conversation / Agent Run and Span ↔ Function / Tool / Step

✅ Specialize for LLMs

  • Added generation = a span optimized for LLMs
    → Knows about input, output, model, usage.tokens

  • Added score = a metric designed for evaluation
    → You don’t have to figure out “where to put quality/accuracy”

💡 Note:
Langfuse only uses the trace schema from OpenTelemetry (OTel) — not the full OTel suite (like logs or metrics).

Whenever it wants to send data to its backend, it uses the OTel SDK for traces, formats the data to the trace schema, and sends it accordingly. It only uses the attributes field in the trace schema to transmit data, applying its own semantics.

📌 Even custom metrics like tokens, scores, etc., are sent via the attributes field as metadata/custom values — not as separate OTel metrics.

✅ Dashboard Built for AI

Instead of Jaeger/Grafana, you get a UI where:

  • 📄 You can read prompts/responses nicely
  • 🔢 Token counts and costs are first-class citizens
  • 📈 Scores (accuracy, helpfulness) are plotted over traces
  • 🔍 You can filter/search traces by use

Inside Langfuse: What It Tracks and How It Works

As we’ve already established, Langfuse uses the OTel’s trace schema/data model for structuring and transporting data to its backend (Langfuse UI).

❓ But how is it done? How is the data actually sent? How does the backend understand it?
Let’s uncover this.

Langfuse leverages the trace schema — attributes field within each span.The OpenTelemetry trace schema allows arbitrary key-value pairs to be embedded inside this field.

For example:

1
2
3
4
5
6
7
8
{
  "attributes": [
    { "key": "prompt", "value": "Tell me a joke" },
    { "key": "tokens_used", "value": 42 },
    { "key": "model", "value": "gpt-4" },
    { "key": "x", "value": 100 }
  ]
}

However, these generic keys like “prompt” or “x” don’t carry any semantic meaning for the Langfuse backend. Instead of raw, arbitrary keys, Langfuse defines a custom naming convention using structured keys. Examples

"langfuse.observation.input", "langfuse.observation.model.name", "langfuse.observation.metadata", and similar keys have special semantic meaning in the Langfuse backend.The data sent under these keys is interpreted specifically As Langfuse puts it:“LLMs and agents are their own category of systems, so we need dedicated semantics for them.” These conventions and objects are understood uniquely by the backend.

1
2
3
4
5
6
7
[
  {"key": "langfuse.observation.input","value": "{\"sql\": \"SELECT * FROM users WHERE active=1\"}"},
  {"key": "langfuse.observation.output","value": "{\"rows\": 42}"},
  {"key": "langfuse.observation.model.name","value": "gpt-4"},
  {"key": "langfuse.observation.metadata.custome.my.value","value": 12345},
  {"key": "langfuse.observation.cost_details","value": "{\"input_cost\": 0.00024, \"output_cost\": 0.00056}"}
]

⚠️ Note:
The structured { key, value } format shown above is what actually goes over the wire in the OTel trace payload.

But for simplicity, when you dump the trace data to your console or logs, the same data is shown in as below. Console Output Format (Flattened Attributes)

1
2
3
4
5
6
"attributes": {
 "langfuse.observation.input": "What is earth?",
 "langfuse.observation.model.name": "gpt-4",
 "langfuse.observation.output": "earth is a planet.",  
 "langfuse.observation.type": "generation",
}

THe langfuse code can be instrumented means the story can be toled in the below way

Core Concepts in Langfuse (trace ,span, generation)

  • Traces (💡 Root span)
    • Represents a complete session, request, or workflow. It’s the top-level context for everything that happens in a user interaction or model execution. It is actually a span but at the top level
    • OTel equivalent: Same as a root span in OpenTelemetry.

      1
      2
      3
      4
      5
      6
      7
      
        with langfuse.start_as_current_span(name="process_request", input={"query": "What is AI?"}) as root_span:
              ...
          # or via observe
      
        @observe()
        def handle_request(user_input: str):
            return f"Processed: {user_input}"
      
    • 📌 Every Langfuse instrumentation begins with a Trace.
  • Spans (a span inside root span i.e trace )
    • Represents a step within a trace — for example, calling a function, querying a DB, retrieving documents, or performing post-processing.
    • Langfuse-specific features:
      • You can use .update_trace() or .update() to log metadata.
      • You can log structured events or scores via .score() or .log_event().

        1
        2
        3
        4
        5
        
          with langfuse.start_as_current_span(name="process_request", input={"query": "What is AI?"}) as root_span:
              with lf.start_as_current_span(
                      name="db_query",
                      metadata={"sql": "SELECT * FROM users"}
                  ) as span
        

⚠️ If a span is the first thing created, it becomes the root span → trace.

📌 Spans are used for everything that’s not an LLM generation.

  • Generations
    • A specialized span in Langfuse that represents an LLM call.Langfuse-only concept.
    • Captures:
      • input , output and model, provider (e.g., OpenAI, Claude) also tokens, latency, cost

        1
        2
        3
        4
        5
        6
        7
        
          with langfuse.start_as_current_span(name="process_request", input={"query": "What is AI?"}) as root_span:
              with lf.start_as_current_generation(
                    name="llm_answer",
                    model="gpt-4",
                    input="What is earth?",
                    usage_details={"prompt_tokens": 12}
                ) as gen:
        

📌 Can be nested inside a trace or a span. Shown in the Langfuse UI with prompt/response formatting.

📌 Mostly used in place of span where there is a LLm interaction as it contains more parameters that can be used to descrive the LLm interaction unlick of a span


Instrumentation in Langfuse

Langfuse provides 3 primary methods to instrument your code. Each supports trace/span/generation creation, but with different control levels and complexity.


1. @observe Decorator — Simple Function Tracing

Wraps a function automatically into a trace/span. Ideal for simple workflows.

1
2
3
4
5
6
7
8
9
10
from langfuse import observe

@observe(
  name="calculate_sum",
  metadata={"langfuse_user_id": "u123", "tags": ["prod"]}
) # Both name and metadata are is optional if name not given it takes the funtion name
def add(a, b):
    return a + b

result = add(2, 3)

🔍 What it does:

  • Automatically creates a trace (if not already active) and Wraps the function call in a span, Span name → calculate_sum
  • Input = function arguments Input → {a: 2, b: 3} , Output = return value Output → 5 (its automatic)
  • Injects context so nested spans inherit from it

2. Manual Spans & Generations — Full Control

Use when you need fine-grained instrumentation, such as wrapping specific LLM calls or nesting multiple steps. When you need more flexibility:Choose whether it’s a span or generation or Explicitly set input, output, metadata, etc.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from langfuse import get_client

langfuse = get_client()

# Manually start a span (or trace if root)
with langfuse.start_as_current_span(name="process_request", input={"query": "What is AI?"}) as root_span:
    span.update_trace(user_id="user-abc", session_id="sess-123")

    # another extra span for a api fetch
    with langfuse.start_as_current_span(name="fetch_data") as span:
        span.update(metadata={"source": "API"})
        span.score(name="success", value=True, data_type="BOOLEAN")

    # Track an LLM generation
    with langfuse.start_as_current_generation(
        name="openai_response", model="gpt-4", input="What is AI?"
    ) as gen:
        # Simulate response
        output = "Artificial Intelligence is..."
        gen.update(output=output)

    span.update_trace(output={"response": output})

langfuse.flush()

🧠 Key APIs:

  • start_as_current_span(...)
  • start_as_current_generation(...)
  • .update_trace(): adds or updates trace-wide data
  • .update(...): updates specific span/gen fields
  • .score(...): logs a score
  • .log_event(...): logs a structured event

3. Drop-in Integrations (LLM Providers)

Langfuse can automatically wrap clients like OpenAI, Anthropic, and others.

✅ Captures inputs, outputs, and token usage — no boilerplate needed.

Example: Instrumenting OpenAI : Langfuse automatically records: 🧾 input (prompt messages), 📤 output (model response), 🔢 usage (token counts).

1
2
3
4
5
6
from langfuse.openai import openai

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Summarize Langfuse"}]
)

Each trace/span/generation can be instrumented with:

FieldHow to SetNotes
inputinput={"x": "y"} in context manager or auto from decoratorTracked for each unit
outputupdate_trace(output=...) or returned value from @observeUseful for prompt responses
user_idupdate_trace(user_id=...)Ties activity to users
session_idupdate_trace(session_id=...)For grouping
metadataupdate_trace(metadata={...}) or @observe(metadata={})Attach custom structured data
tagsmetadata={"tags": [...]}Used in UI and filtering

Scoring, Feedback & Events and First-Class Atributes in Langfuse.

Langfuse v3 gives you a few semantics (special attributes that the SDK + backend + dashboard understand): The SDK automatically process and sent in the attribute filed of trace

  • 🟡 Score → Structured evaluation object via .score()
  • 🔢 Tokens → Tracked as usage_details inside a generation usage_details={"prompt_tokens": 10, "completion_tokens": 25}
  • 💰 Cost → if you provide model, it looks up token pricing; else you can set manually as metadata
  • ⏱️ Latency → Measured automatically from span/generation start/stop

🟡 Score a trace/span

1
span.score(name="accuracy", value=0.9, comment="Looks good")

🟢 Log an event (for debugging or analytics)

1
span.log_event(name="doc-retrieved", level="info", payload={"doc_id": 123})

Note These metrics are sent to the backend via OTel trace and via the attribute field

What About Your Custom Fields (info metadata)?

If you want to log arbitrary extra info like:

1
2
my_custom_thing = "my_value"
custom.my.value = 12354

You can safely store them in:

  • 🗂️ metadata → Structured JSON dict (unlimited depth)
  • 🔖 tags → Flat list of strings

✅ These won’t break anything
✅ They will appear in the Langfuse UI under “Metadata” or “Tags”
❌ But they won’t be plotted automatically like scores, tokens, or latency

Langfuse Dashboard will render them just as Metadata


Reminder: Instrumentation is Manual by Design

Langfuse does not automatically “see” your inputs, outputs, or model calls. same as OTel You are responsible for:

  • Starting and stopping spans/generations
  • Logging inputs/outputs
  • Logging token usage, cost, latency (optionally)
  • Scoring and tagging

Practicals and Examples

Note You need to .flush() as langfuse traces are async and the flush safly sends all the unsend data in the event loop

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
from langfuse import get_client

lf = get_client()

# Start a trace (root)
with lf.start_as_current_trace(
    name="pipeline",
    metadata={"pipeline_id": "123", "experiment": "A1"},
    tags=["v3", "demo"]
) as trace:

    # Generic span
    with lf.start_as_current_span(
        name="db_query",
        metadata={"sql": "SELECT * FROM users"}
    ) as span:
        # You can update metadata later
        span.update(metadata={"rows_returned": 42})
        
        # Add a score to this span
        span.score(
            name="db_latency_ok",
            value=True,
            data_type="BOOLEAN",
            comment="Returned fast"
        )

    # LLM Generation (specialized span)
    with lf.start_as_current_generation(
        name="chat_step",
        model="gpt-4o",
        input="What is Langfuse?",
        usage_details={"prompt_tokens": 15},  # tokens
        metadata={"agent": "faq-bot", "custome.my.value": 12354}  # 👈 custom field
    ) as gen:
        gen.update(
            output="Langfuse is an observability tool for LLMs",
            usage_details={"completion_tokens": 60},  # more tokens
            cost_details={"input": 0.0015, "output": 0.0030}  # manual cost
        )
        
        # Add evaluation score
        gen.score(
            name="helpfulness",
            value=0.8,
            data_type="NUMERIC",
            comment="Good but not perfect"
        )

# Push all data
lf.flush()
examples of real data dumps : [CLICK ME]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
  -> breakpoint()
  (Pdb) c
  // This is A normal span=============================================
  {
      "name": "db_query",
      "context": {
          "trace_id": "0xfa5f4e5915575785b27bbb3f4b1e29a8",
          "span_id": "0x146d16d10dc60db3",
          "trace_state": "[]"
      },
      "kind": "SpanKind.INTERNAL",
      "parent_id": "0xd73a74fa5f51c7df",
      "start_time": "2025-08-24T18:26:28.179889Z",
      "end_time": "2025-08-24T18:26:28.180541Z",
      "status": {
          "status_code": "UNSET"
      },
      "attributes": {
          "langfuse.observation.input": "{\"sql\": \"SELECT * FROM users WHERE active=1\"}",
          "langfuse.observation.output": "{\"rows\": 42}",
          "langfuse.observation.metadata.db": "users",
          "langfuse.observation.type": "span",
          "langfuse.observation.level": "INFO",
          "langfuse.observation.status_message": "Fetched 42 active users",
          "langfuse.observation.metadata.my_new_metric": "Somevalue"
      },
      "events": [],
      "links": [],
      "resource": {
          "attributes": {
              "telemetry.sdk.language": "python",
              "telemetry.sdk.name": "opentelemetry",
              "telemetry.sdk.version": "1.36.0",
              "service.name": "unknown_service"
          },
          "schema_url": ""
      }
  }
  > /home/abhishek/workspace/agent_fabric/testing_main.py(50)<module>()
  -> breakpoint()
  (Pdb) c
  This is Generation Example=============================================

  {
      "name": "llm_answer",
      "context": {
          "trace_id": "0xfa5f4e5915575785b27bbb3f4b1e29a8",
          "span_id": "0xa37d33a889de4110",
          "trace_state": "[]"
      },
      "kind": "SpanKind.INTERNAL",
      "parent_id": "0xd73a74fa5f51c7df",
      "start_time": "2025-08-24T18:26:35.394530Z",
      "end_time": "2025-08-24T18:26:35.395176Z",
      "status": {
          "status_code": "UNSET"
      },
      "attributes": {
          "langfuse.observation.input": "What is earth?",
          "langfuse.observation.model.name": "gpt-4",
          "langfuse.observation.output": "earth is a planet.",
          "langfuse.observation.usage_details": "{\"completion_tokens\": 28}",
          "langfuse.observation.metadata.agent": "fake-custome-agent",
          "langfuse.observation.metadata.custome.my.value": 12345,
          "langfuse.observation.type": "generation",
          "langfuse.observation.cost_details": "{\"input_cost\": 0.00024, \"output_cost\": 0.00056}"
      },
      "events": [],
      "links": [],
      "resource": {
          "attributes": {
              "telemetry.sdk.language": "python",
              "telemetry.sdk.name": "opentelemetry",
              "telemetry.sdk.version": "1.36.0",
              "service.name": "unknown_service"
          },
          "schema_url": ""
      }
  }
  This is root-span completed=============================================

  {
      "name": "root_pipeline",
      "context": {
          "trace_id": "0xfa5f4e5915575785b27bbb3f4b1e29a8",
          "span_id": "0xd73a74fa5f51c7df",
          "trace_state": "[]"
      },
      "kind": "SpanKind.INTERNAL",
      "parent_id": null,
      "start_time": "2025-08-24T18:26:28.179384Z",
      "end_time": "2025-08-24T18:26:35.396454Z",
      "status": {
          "status_code": "UNSET"
      },
      "attributes": {
          "langfuse.observation.input": "{\"intent\": \"user_asks_question\"}",
          "user.id": "user_123",
          "session.id": "sess_abc",
          "langfuse.trace.tags": [
              "demo",
              "pipeline",
              "v3"
          ],
          "langfuse.observation.metadata.pipeline_version": "0.1",
          "langfuse.observation.metadata.team": "aget-team",
          "langfuse.observation.type": "span",
          "langfuse.observation.level": "INFO"
      },
      "events": [],
      "links": [],
      "resource": {
          "attributes": {
              "telemetry.sdk.language": "python",
              "telemetry.sdk.name": "opentelemetry",
              "telemetry.sdk.version": "1.36.0",
              "service.name": "unknown_service"
          },
          "schema_url": ""
      }
  }

🚫 What You Cannot Do

  • ❌ You cannot invent new object types beyond Span, Generation, or Event.
  • ❌ You cannot define a new schema like my_custom_metric outside of .score().
  • ❌ You cannot push OTel metrics/logs — Langfuse is trace-only.
  • ✔️ Custom values (e.g., custome.my.value) must go in metadata, not as first-class objects. like score,token,cost ,latency etc
This post is licensed under CC BY 4.0 by the author.