I was reading Sandy Dunlop's post on a Fast String Reader implementation with some test statistics.
It's a great post, but some things really struck out at me: the string.IndexOf(...) and the string.SubString(...) calls in the middle of a loop.
I've been trained (poorly -- by VB6) that string manipulation is extremely costly and will kill your performance. I believed that it's best done with unsafe() pointers, and, barring that, with arrays. I *think* I'm still right on the unsafe() pointers, but I'm definitely NOT right on the array calls.
Ayende recently proved that String.IndexOf is really fast. My next target was SubString(). This method just CAN'T be fast. Well, after playing around with Sandy's code, I can't find any other non-unsafe() way of getting a sub-string of a string that is anywhere NEAR as fast as String.SubString().
Lesson? Most of the code for System.String is unmanaged and is optimized C/C++ code. It's also very WELL TESTED code that has survived the test of time in the wild. Assuming you can, or worse yet, attempting to do so, will result in much worse performance.
In fact, most of the code in the entire BCL is optimized and well tested, having survived quite awhile in the wild under attack. You should first assume that a given algorithm or function in the BCL is a best-of-breed implementation until you've proven otherwise. THEN you may optimize and then you MUST contribute that to the community :)
P.S.- Surprisingly, the Array.IndexOf function on the char[] produced from a stringInstance.ToCharArray() method is far slower than using the stringInstance.IndexOf() directly.
P.P.S. - Apparently Array.IndexOf() hasn't undergone the intense performance scrutiny that String.IndexOf() has.