The shell is not a house and the turtle cannot leave it. The carapace is formed from expanded and fused ribs and vertebrae covered by keratin scutes, and the shoulder girdle sits inside the ribcage, an arrangement found in no other vertebrate and one of the more striking rearrangements in the fossil record. The animal is welded to it.
Turtles are also genuinely long-lived. Several tortoise species routinely exceed a century, and individual Aldabra and Galapagos tortoises have well-attested ages beyond 150 years; Jonathan, a Seychelles giant tortoise on Saint Helena, is documented from at least 1882. Some species show negligible senescence, meaning the annual probability of death does not rise measurably with age. The longevity symbolism is therefore not a folk exaggeration; it is an accurate observation, and Chinese and Japanese use of the turtle as an emblem of long life rests on something real.
The turtles all the way down story deserves a note because it is told constantly and sourced almost never. The anecdote, a lecturer is told by an old woman that the world rests on a turtle, asks what the turtle rests on, and is told it is turtles all the way down, is attached variously to William James, Bertrand Russell, Thomas Huxley and others, with no reliable attestation for any of them. Its earliest printed forms are nineteenth-century and already anonymous. Stephen Hawking used it to open A Brief History of Time, which is where most people met it.
The underlying image of stacked world-bearing animals does appear in some South Asian accounts, but the infinite regress is a Western philosophical joke, and it is not a summary of anyone's cosmology.