Soap bubble foam made with helium floats up, but due to extreme fragility hardly counts as "material". There are many solid foam materials though - PUR foam, or styrofoam to name the most common. They typically use carbon dioxide for inflation though (usually produced from precursors of the foam, as a desirable side effect of their reaction).
But it shouldn't be too difficult to make solid foam filled with helium (or hydrogen) in proportions assuring positive buoyancy in air, and I can imagine desirability of it, at least as a filler in applications where mass costs a premium (transport, aviation) even if its structural properties were to be too poor for any other purpose.
Is such material produced? Is it used anywhere? Or if not, why?