How to dynamically update the chat callback handler based on session?

Panel
1

I have an existing Holoviz Panel application that uses a Langchain agent.

I want to instrument calls to langchain (prompt and output), using ‘LangFuse’ which offers a callback handler that can be plugged into standard langchain callbacks.

However I can’t easily pass in a Panel session ID to link conversations, because the ChatInterface and LLM handler seem to be global to all sessions (assuming I’m using the correct terminology).

For example:

  • using the pn.state.on_session_created handler I can get the session ID when a new user starts using the app, and update the handler
  • but when a second user joins, the handler will be updated to their session ID
  • so the conversations for both users will now be attached to the second session ID
# ...other set up here (omiitted)

langfuse = Langfuse()
langfuse_handler = CallbackHandler() # init langfuse handler

# langchain agent
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    callbacks=[langfuse_handler] # set langfuse callback handler
)

chat_interface = pn.chat.ChatInterface(
    callback=agent_executor,  # set agent callback handler
    # ...other chat interface settings
)

  def session_created(session_context):
      # how to update agent_executor with session id?

pn.state.on_session_created(session_created)

pn.serve(# ...serve settings)
2

Based on your title, I think you can set

chat_interface.callback = ... to dynamically update it

3

Hey @ahuang11, just to clarify my question isn’t how to change the callback. Its how to provide the session ID to the callback in a way that isn’t overwritten when a new session starts.

My confusion is the chat_interface variable is global, and so is the callback handler. Is there a way to scope them to the session?

4

user_info maybe Accessing User information — Panel v1.4.2