The reason is that when the stem (or branches) of the tree are shaken, they come in motion, while the fruits due to inertia, remain in the state of rest. Thus, the massive and weakly attached fruits get detached from the branches and the fall down due to the pull of gravity.