Kiwi:
You're absolutely right. Every lens, pro-level or otherwise, has its sweet spot in terms of aperture setting. Every lens in your bag should be tested to find that best setting. For most lenses that I've tested, it seems to be around f/8 to f/11 as you have also discovered.
There are many more factors, however, that need to be considered in going for the sharpest images you can get. One of the most important is understanding how to use a tripod. Some folks I go out with simply mount the camera on the tripod and shoot. They seem to think that simply using a tripod is all that is needed. How they use it is what they need to understand. One needs to take into account the focal length, shutter speed and how steady the tripod/head is given the shooting parameters. For example, a long lens, such as a 500mm is extremely difficult for me to get sharp photos at anything less than 1/400th of a second. At that focal length, at that speed, the slightest breeze seems to turn my images very soft. (So, there are other factors - environmental and weather conditions.) Anyway, a 200mm lens, however, has no problems even at very slow shutter speeds. So, practice how you steady the lens even when it's on the tripod - it's important.
Use a cable release and mirror lockup. I used to shoot a lot with my 10D and the Canon 100-400mm lens on a heavy aluminum tripod. I found, after months of frustration, that the aluminum tripod would vibrate horribly with the mirror slap, and that simply pressing my shutter button would cause movement. Not a problem at very high shutter speeds, but starting in the range of 1/400 and below, shots were blurred. I finally figured out that, whenever possible, I should at least use a cable release and, for slower speeds, lock up my mirror. Tough to do with an active subject! Then I discovered the wonderful performance of a carbon fiber tripod. Man, did that cure a lot of ills!
Automatic focusing isn't always the answer. I have never really trusted autofocus. Most of the time it seems to work, but for really critical focusing (that bird's eye at a bit of a distance, for example), I still tend to rely on manual focus. Sometimes, you can actually be faster doing that. I found that to be particularly true with the 10D and 20D, especially in low light.
Always shoot multiple frames. Often, the first shot in a series is soft because of camera movement when you pressed the shutter button. But I have found that if you continue to hold the shutter button down for several frames, the 2nd or 3rd shot can be sharper than the first one. That is, if you're doing the other stuff properly.
Noise level. Now, I know the old addage is just get the shot, no matter what. Well, I found that excess noise just absolutely ruins the picture for me. The 10D gets really noisey at ISO 400 and above especially if your lighting and other factors are not dead on. So, here is one case where knowing your equipment can save a lot of frustration. Knowing my ISO limitations on my 10D, and my distinct aversion to noise, I simply know I must shoot at nothing above ISO 200. That, of course, limits my shooting to great light and makes it really tough to get my shutter speeds where they need to be. But, for me, looking at a very noisey image makes me physically ill. May be some kind of OCD!
The point to all this is to do just what you did - experiment. Only through constant attention to the various factors and understanding how they affect your work can you possibly hope to obtain the best results. Oh, and shoot a lot, judge results objectively, and then go shoot some more.
Paul
Paul Wolf
www.paulwolf.smugmug.com