Gimp makes it really easy to create multi-res .ico files. In fact, this is one of the few areas where Gimp beats Photoshop hands down.
The trick is to use multiple layers with different sizes.

Here are the basic steps.

1. Open your image in Gimp
2. Make your canvas square
3. Resize your layer to the image
4. Scale the layer to the largest size in your .ico file like 64 pixels
5. Duplicate the layer
6. Scale the duplicate layer to the next size
7. Keep duplicating / scaling for all the sizes you want in your .ico file
8. Save as .ico

That's it. Watch the video to see all these steps in action.
I have purchase several .ico editors on Windows that are not as good as the Free Gimp.
Whether you use Gimp for photo editing or are an avid Photoshop fan, you should still install Gimp for this feature alone!