How to use GPT vision with HoloViews

Showcase
1

Just a minimal example to help getting started (for myself in the future too :D)

import base64
from io import BytesIO
import holoviews as hv
import openai

curve = hv.Curve([0, 1, 2])
with BytesIO() as f:
    hv.save(curve, f, fmt="png")
    f.seek(0)
    base64_image = base64.b64encode(f.read()).decode('utf-8')


client = openai.AsyncClient()
await client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What’s in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{base64_image}"},
                },
            ],
        }
    ],
)

content = response.choices[0].message.content

Outputs: 'The image shows a simple linear graph with a line that extends from the origin (0,0) to (2,2). The x-axis ranges from 0 to 2, and the y-axis also ranges from 0 to 2. The line appears to have a positive slope, indicating a linear relationship between the variables x and y, specifically following the equation \\(y = x\\).'

1 Like