Ability to send both text and file attachement intput together, in ChatInterface?

Alex-Wenner-FHR · May 7, 2025, 3:29pm

I have two widgets in my ChatInterface, [pn.widgets.TextInput(...), pn.widgets.FileInput(...)] However I have noticed that I cannot send both a text message and file attachment together.
When I type text and switch to the file input tab, it automatically sends the text message and vice versa.

I would like to achieve this given that LLMs are multimodal and can handle text and images together for example. Any advice on how to do this? I have looked for some examples of this but cannot find any! Thanks!

ahuang11 · May 7, 2025, 5:56pm

Thanks! AFAIK I don’t think this is possible yet so this would be a very good feature request to have on GitHub

Alex-Wenner-FHR · May 7, 2025, 6:19pm

github.com/holoviz/panel

Ability to send text and attachment together in ChatInterface

opened 06:19PM - 07 May 25 UTC

Alex-Wenner-FHR

## Description I have two widgets in my `ChatInterface`, `[pn.widgets.TextInput(…...), pn.widgets.FileInput(...)]` However I have noticed that I cannot send both a text message and file attachment together. When I type text and switch to the file input tab, it automatically sends the text message and vice versa. I would like to achieve this given that LLMs are multimodal and can handle text and images together for example. ## Suggested Solution A solution that would essentially pair text + attachment. The attachment renderers are great. It would be awesome to see a text string + csv uploaded together the UI would reflect a chat message then the rendered mime type. Maybe the underlying contents could be that of a list? `[text, mime_type_attachment]` or something to that effect. That way under the hood, say in a callback, when you grab that content to send to an LLM you can distinguish the text string vs the attachment. ## Considered alternatives I have tried to implement something like this manually through callbacks and such, but can't quite get it to look as good as the renderers do nor be able to handle them together as one message. ## Additional context https://discourse.holoviz.org/t/ability-to-send-both-text-and-file-attachement-intput-together-in-chatinterface/8742 Something like this, where I send a text but I also attach a CSV. They could be published as the same message or two consecutive, whatever works. But then on the backend of things the input is some sort of object that pairs them together, like ```python [text, attachment] | {"text": <input>, "attachment": <mime_type>} | etc. ``` <img width="328" alt="Image" src="https://github.com/user-attachments/assets/ad28d3d0-51a1-4f72-a9de-c3daf8f343c5" /> <img width="300" alt="Image" src="https://github.com/user-attachments/assets/c2f583d5-e539-4631-aa59-30f18670e1f7" />

@ahuang11