How do I make ipympl the Matplotlib backend when using HoloViews?

When using HoloViews the Matplotlib backend seems to get set by default to Agg and I can’t figure out how to set it to an interactive backend. Is there a way to do that?

The following seems to work for Matplotlib on its own, but has no effect on figures rendered with hv.output(... backend='matplotlib'):

import holoviews as hv
import matplotlib as mpl
hv.extension('bokeh', 'matplotlib'
%matplotlib ipympl

What is the correct way to use ipympl as the Matplotlib backend for HoloViews.