Stream text to ChatInterface?

Trying to wrap my head around ChatInterface.
Use case: a fairly complex dialgue, with steps controled by python code.
I want to capture streaming output from the chatbot, parse it on the fly,
and stream a modified version of that output to a chatinterface instance for display.

Something like

import asyncio
import nest_asyncio
nest_asyncio.apply()

async def stream_text(chat_interface):
    for i in range(10):  # Simulate sending 10 messages
        message = {"text": f"Message {i+1}", "user": "bot", "avatar": "https://via.placeholder.com/24"}
        # Directly append the message dictionary to the chat_interface's value list
        # WHAT GOES HERE???   chat_interface.value.append(message)
        await asyncio.sleep(1)

pn.chat.ChatInterface(height=400, width=400).servable()

await stream_text(chat_interface)

I.e., the user provices a prompt (pushed to chat_interface, and sent to chatbot)
the streaming chatbot response is parsed, modified, and the modified stream is sent to the chat_interface.

  • How can I achieve this?
  • Any better way to achieve it than what I am currently pursuing?

To stream text, it’s either using yield or instance.stream

Have you taken a look at ChatFeed — Panel v1.4.0rc2?

Or Basics - Panel Chat Examples

I see. So my use case could be built using

chat_feed = pn.chat.ChatFeed()
chat_feed

and

message = None
for n in "123456789 abcdefghijklmnopqrstuvxyz":
    message = chat_feed.stream(n, message=message, user="Parrot", avatar="🦜")

message = None
for n in "9765":
    message = chat_feed.stream(n, message=message, user="Dog", avatar="🐕")

@ahuang11 Thank you!

While that works, it does not let me customize the appearance of the ChatMessage.
I tried reading the docs and the code, but am currently stuck on the following:

  • How do I insert a ChatMessage with
    default_avatars = {"System": "☂️", "User": "👤"}, reaction_icons={},
    show_user       = False,
    show_timestamp  = False,
    show_copy_icon  = False,
)

that I can then stream data to?

  • How do I get the chatFeed display to scroll automatically such that the latest message stays in view?
  • How do I get the avatar to line up with the top of the message?
    When the message is a single line, the avatar extends way beyond the text.
    Also, can I resize it somehow? It’s rather overwhelming!
  • Is there a way to print the dialogue from the chatFeed display?

I recommend trying out pip install panel>=1.4.0rc2 Lots of fixes in it

This too: ChatMessage word break on long continuous words by ahuang11 · Pull Request #6509 · holoviz/panel · GitHub

See ChatFeed — Panel v1.4.0rc2 and ChatMessage — Panel v1.4.0rc2 for styling. Note these are only valid with Panel>=1.4.0

1 Like

Given chat_feed = pn.chat.ChatFeed() and message = pn.chat.ChatMessage( user="System"),

how do I add message to chat_feed for display?

Or rather: I want to add a ChatMessage to ChatFeed and stream data to it.
Is this the way to go?

message = pn.chat.ChatMessage( user="System")
chat_feed._replace_placeholder(message)
txt = ""
for n in " I see - will do":
    txt += n 
    message.update(txt)

As for styling, I arrived at

path_to_stylesheet = """
    .avatar {
        background-color: #eeeeff;
        border-radius:50%;
        margin-top: 0;
        margin-left: 0;
        margin-right: 0;
    }
"""
message = pn.chat.ChatMessage( user="System", avatar= "☂️", stylesheets=[path_to_stylesheet],
                                 show_user       = False,
    show_timestamp  = False,
    show_copy_icon  = False,
    sizing_mode="stretch_width",
)
chat_feed._replace_placeholder(message)
message.margin=(0,0)

which takes a noticeable amount of time to execute. :slightly_frowning_face:

While a lot better given what I need, that still leaves 3 issues:

  • I need to scroll down manually to see the latest inserted message
  • Successive messages are still too far apart. I have not yet found the css setting to reduce that spacing
  • I’d still like to reduce the size of the avatar!

I also believe there must be a better code solution - I do however want to separate the GUI from any processing.

Any comments, anybody?

Something like this i think…

import panel as pn
chat_feed = pn.chat.ChatFeed()
message = pn.chat.ChatMessage(user="System")
chat_feed.send(message)
for i in range(0, 12):
    chat_feed.stream(f"new message {i}", message=message)
  • auto_scroll_limit (int): Max pixel distance from the latest object in the Column to activate automatic scrolling upon update. Setting to 0 disables auto-scrolling.

You might want to check out this in progress revamp for styling: Revamp and modernize docs completely for 1.4.0 by ahuang11 · Pull Request #136 · holoviz-topics/panel-chat-examples (github.com)

Also this:
ChatMessage — Panel v1.4.0rc2 (holoviz-dev.github.io)

(requires 1.4.0rc)

I thought that is how it is supposed to work.
It however hangs in chat_feed.stream() on my system, without ever updating the message. (with both the official panel release and with 1.4.0rc)

Re styling: found it finally! set height and min-height on .left
Re scrolling: updating text in a text message: once the text gets too long,
scolling stops. I would prefer seeing the end of the message without having to scroll manually. Can that be achieved?

To recapitulate, the following code does work:

message_params = dict(
    default_avatars = {"System": "☂️", "User": "👤"}, reaction_icons={},
    show_user       = False,
    show_timestamp  = False,
    show_copy_icon  = False,
    sizing_mode     = "scale_width",
)

chat_feed = pn.chat.ChatFeed(message_params=message_params,
                             height=300, max_width=2200)
chat_feed

followed by

path_to_stylesheet = """
    .avatar {
        background-color: #eeeeff;
        border-radius:50%;
        margin-top: 0;
        margin-left: 0;
        margin-right: 0;
    }
    .left {
        background-color: lightblue;
        min-height: 50px;
        height: 50px;
    }
"""
message = pn.chat.ChatMessage( user="System", avatar= "☂️", stylesheets=[path_to_stylesheet],
                               show_user       = False,
                               show_timestamp  = False,
                               show_copy_icon  = False,
                               sizing_mode     = "stretch_width",
)
chat_feed.send(message)
message.margin=(0,0)

txt = ""
for n in "Updating the message works, streaming to it does not....":
    txt += n 
    message.update(txt)

1 Like

Thank you for sharing! Perhaps this can be documented too (reducing spacing) :slight_smile:

@ahuang11
I think this should be an example:

  • show how to create a minimal example that uses ChatFeed (or ChatInstance),
    or a new class as a GUI only. I.e., all user system interactions are carried out
    in separate code, with the GUI used to display the messages only.
  • show how to remove all the extra ChatMessage features,
    and tighten up the spacing.
    The avatars are still too large, and so is the spacing
    between the avatar and the message
  • I am currently struggling with adding some buttons and a larger text input area.
    I am running into sizing an alignment issues.
  • Given how much effort it turned out to be trying to put this together,
    I think it should definitely be documented.

Any suggestions on how to put this together?
Incidentally, the default_avatars in message_params does not seem useful in this context. Maybe ChatFeed should have an append_message() fn
that one can stream data to? I have not been able to make that work,
Instead writing

def add_message( chat_feed, avatar = "☂️"):
    message = pn.chat.ChatMessage(
                 avatar=avatar, stylesheets=[STYLE],
                 show_user       = False,
                 show_timestamp  = False,
                 show_copy_icon  = False,
                 sizing_mode     = "stretch_width",
    )
    message.margin=(0,0)
    _ = chat_feed.send(message)
    return message

and using

txt = ""
for n in "Updating the message works, streaming to it does not.... ":
    txt += n 
    message.update(txt)

to change the displayed text.
I’d love to know what you think!

I found a bug; if object=None, it won’t stream, but this works

import time
import panel as pn


def callback(event):
    for i in range(0, 12):
        time.sleep(0.05)
        chat_feed.stream(f"new message {i}", message=base_message, replace=True)


chat_feed = pn.chat.ChatFeed()
base_message = pn.chat.ChatMessage("")
chat_feed.send(base_message)

button = pn.widgets.Button(name="Start", button_type="primary")
button.on_click(callback)
pn.Row(
    pn.Column(button, chat_feed)
).show()

Export-1710790699008

Fixed here:

1 Like

Yet another puzzle I have not yet solved: .message does not fully use the horizontal extent of .center …

Maybe .right?

unfortunately no, .right is beyond .center which I showed in black.
I can’t get .message to use all the available space in .center

In fact, it looks like .center is contained in .right?!

Do you mean this part?

.center is indeed contained in .right.

Maybe margin-right