Observability for LLM Apps Without Drowning in Logs

If you only log HTTP status codes, you will never know whether users stopped trusting answers last Tuesday.

Correlate request ID → retrieval IDs → model version → latency → user feedback. That chain is the unit of debug.

Sample raw prompts and responses carefully. Privacy and storage costs both punish naive “log everything”.

Alert on distribution shifts: sudden empty retrieval, cost spikes, timeout rate, or thumbs-down rate — not only process crashes.

Dashboards should answer “is the product useful?”, not only “is the pod alive?”.