"This tradition goes through Miss Marple and Murder She Wrote — that amateurs would have this arcane, genius talent as if they were musicians or mathematicians or something. It's a hilarious idea if you stop and think about it."
Well, not really. The amateur detective appeared at a time - the nineteenth century - which saw many instances of amateurs/outsiders outwitting the professionals. Darwin was not a biologist. Pasteur was not a doctor. Early egyptologists/archaelogists like Maspero or Schliemann were self-taught. The idea that an amateur detective could challenge and outperforming the coppers thus made sense, especially given everyone's low regard for the official police.
Romanticism was also a key influence, with its notion that the Poet as some kind of a superman who can feel and see things invisible to mere mortals. Dupin is as much an embodiment of that ideal (and maybe more so) than of reason. Same goes for Holmes who, for all his professed rationalism, looks and behaves much like your standard romantic.
Of course in our highly specialized and less than romantic times, the whole stuff sounds, in Sims's words, "hilarious". But looking absurd now doesn't mean it was absurd then.