Yes, all children need their own child passport to travel abroad. It can take a bit of organising but the passport will be valid for five years, even though their appearance is likely to change incredibly.
The application forms are available from a post office branch that offers the passport check and send service. If getting to a post office with your baby is difficult, you can complete the application form online. It will be sent to you to sign and return with the supporting documents. Or you can request a form online and it will be sent to you in the post.
More information can be found by visiting the following government website link.
The Home Office Identity & Passport Advice line can be contacted on 0300 222 0000. Their opening hours are stated as Monday to Friday 8am to 8pm and on weekends 9am to 5:30pm. This may be subject to change.