The tradition attributes its founding to Anan ben David in eighth-century Babylonia, in the aftermath of a disputed succession to the exilarchate around 760 in which, on the standard account, he was passed over and led his supporters out.
That account is disputed. Scholars including Leon Nemoy have doubted that the succession story happened as told, and modern research treats Karaism as the coalescence of several earlier anti-rabbinic currents during the Gaonic period, between the seventh and ninth centuries in Baghdad, attached to Anan retrospectively as a founder. The movement is better understood as a channel that gathered existing dissent than as one man's secession.
The break with the rabbinic mainstream was made permanent by opposition. Saadia Gaon, in the tenth century, attacked Karaism with sustained ferocity, and the effect of his polemic was to make the division formal and lasting rather than to resolve it. Karaism nonetheless flourished: at its height, around the tenth and eleventh centuries, it may have accounted for something like a tenth of world Jewry, with communities from Spain to Persia, and it produced substantial scholarship: Jacob al-Qirqisani's survey of Jewish sects in the tenth century, Judah Hadassi's twelfth-century compendium containing the earliest formulation of Karaite articles of faith, and later codes of law.
Rabbinic authorities have never agreed on a single status for Karaites, treating them variously as Jews of doubtful lineage or as Jews without qualification. In Israel, where the Chief Rabbinate controls personal status, that unresolved question bears directly on marriage and burial.