All of the keyboards I'm familiar with require stereo headphones that have a 6.3mm (a.k.a. 1/4-inch) audio plug. If your headphones have a smaller plug then you'll need to use an adapter with it.
But I'm guessing that the problem you're having is being caused by the type of plug, rather than-- or in addition to-- the size of the plug. When you look at the plug you should see two rings that separate the plug into three sections (i.e., a "TRS" plug)-- not one ring (which would be a "TS" plug), and not three rings (which would be a "TRRS" plug).
As I understand it, iPhones and such commonly use the 3-ring TRRS type of plug, which is why I'm guessing that that's causing your problem. If you purchase an adapter for it, you'll need to buy a male 3.5mm TRRS to male 6.3mm TRS adapter.