Who should decide what features WordPress gets next?

It seems that some people don’t feel the community gets enough input in to what ‘s happening in WordPress. Is that a valid point, or are they just not stepping up to the plate in Trac, IRC, and on the Make blogs?