The old crow was the first crow the Scarecrow had met to see him for what he was: a simulacrum of a Munchkin. Upon this realization, the Scarecrow no longer had any effect upon his field, and therefore, did not give much thought into leaving his post to follow Dorothy Gale. Though his lack of brains may have helped.