Most of the current successful systems for automatic speech recognition are based on hidden Markov models (HMMs). HMMs are basically general-purpose statistical pattern matchers, but have so far proved more successful than approaches which have been based on specific knowledge about speech. This paper discusses the likely reasons for this success. It is argued that, in addition to providing a tractable mathematical framework with straightforward algorithms for training and recognition, HMMs have a general structure which is broadly appropriate for speech: both shortterm spectral variability and temporal variability can be modelled. This general structure can be tailored to known characteristics of the sounds being modelled, but the actual parameters are optimized based on training data. Another very important characteristic of HMMs is that they provide a complete model which can also be employed at higher levels (such as syntax), with only a single decision being made based on finding the best match at all levels. All these aspects combine to make HMMs a powerful approach to speech modelling. It is therefore argued that, in order to progress recognition capabilities further, the best approach is to retain all these advantages of the general framework but to overcome the limitations of the current HMM formalism by making it segment-based rather than frame-based.