Riddonkulous

I'm green then I'm brown. I start high and then fall down. My flesh thick, my milk sweet, My shell like concrete. My name ends wrong, I've been a fruit all along.