We got a couple pairs of "Califone Animal Listening First Preschool Headphones". A web search will reveal several retailers. These are the right size for preschoolers, and they're very tough. In two years of use, a couple of the "ears" of the animal figures have broken off, but this is cosmetic only; the headphones still work well.
These are wired headphones, which means you've got to plug them in at the third row. An extension cable (available at Radio Shack or elsewhere) may be useful if the kid sits in the 2nd row. Wired headphones have the inconvenience of the wire, but they don't have the inconvenience of running out of batteries. A reasonable tradeoff.
Since they're wired, we can use them outside the van, with anything that takes a standard 1/8 stereo headphone jack, such as an iPod or a portable DVD player. They're cheap enough that we got a couple of pairs, so we can keep one dedicated in the van and another for the house.
These are cheap and sturdy, designed for use in schools and other institutions where they take a beating.
There's a volume control on the cord. There's also a volume control by the third row jack where you plug the headphones in. A parent can set the volume at the third row jack to limit the maximum volume, and then you can let the kid play with the control on the headphone cord.
If you insist on wireless, the Odyssey uses fairly standard IR scheme, so there are plenty of headphones that will work. But I'm not aware of any that are designed for small kids. Hmm, seems like there's a business opportunity here for someone...