Yes, I could buy a wood burning stove, but aren't they sort of
depressing? I mean, you don't get to watch the fire, right? I know
there's a pollution downside to burning wood, but I'm not going to be a
big wood burner, just an occasional small, short fire for the hell in it
and a quick warm up.

Depends on the stove. You can get wood-burners with glass doors.
And you can "repair" the chimney by running zero-clearance stovepipe
up it, in which case, the brickwork is purely decorative.