The scarecrow Field I believe is from the Far Away Lands (Wizard of Oz) sub map that you get by walking north in the Harbor.
There are 2 underground sub maps, The Center of The Earth and the Pyramid. One is reached from near the snowy zones and one is reached from in the Desert.
If something is directing you to those then it should tell you exactly where to go.
Plus those sub maps (there are 7 in this game, The Scientists Castle, Underwater, Far Away Lands, Pyramid, Center of the Earth, Wonderous Lands and Pirate Island) all have an initial quest to get you into them and therefore have a great big obvious ! somewhere on the main Map where to start when one is able to start (most require a certain Level to be reached first).