Argthtjtr

There is no such rule, and for that matter, there is no rule about the position of Mount either; there's Rocky Mount in my part of the world, and many Appalachian peaks are known as Nnn Mountain. It is a matter of convention. The same goes for many other geologic or hydrologic features: the Leyte Gulf but the Gulf of Mexico, Loch Lomond but Alemoor Loch, the Isle of Wight but Portsea Island.

That said, Nnn River is the far more prevalent form in the U.S.; it would be quite rare to hear of the River Missouri or the River Columbia in prose. This is also true of other American terms for streams, e.g. branch, brook, run, kills (though not so much for lakes). This format may then carry over to foreign names, unless the entire foreign name is borrowed in whole: thus Americans usually known the Rio Amazonas as the Amazon River, but it is the Arroyo de la Laguna east of San Francisco Bay and not the Laguna Arroyo.

As JeffSahol noted, you can somewhat sidestep the question for rivers by referring to them with the definite article: the Hudson, the Platte, the Potomac.

I suspect British English tends to put river (and the definite article) before the name. It’s always the River Thames, and not Thames River. Mountains are sometimes preceded by Mount, sometimes not. It’s always Mount Snowdon, but it's Ben Nevis (not Mount Ben Nevis) and the Eiger (not Mount Eiger).

Interesting...note that rivers are denoted by the definite article "The", though. The Colorado, as distinguished from Colorado, the state. Mountains' names, without the definite article, tend to need their "honorific" to distinguish them from other proper names or ordinary nouns...with exceptions, as noted below in the comments.

Most rivers in England are "River Foo" but there are exceptions e.g. "Moors River". Streams are always "Foo Stream". Foreign rivers are variously called "Foo River", especially if the author is N American, "River Foo" if the author is e.g. British, but most often just "the Foo" or "the Foo river". This information is gleaned from various book-based searches of river names at Google Books.

The difference, it seems to me, is that Old World rivers have their own names (often personified as a god or nymph) but colonists typically named rivers after something (e.g. Hudson River after Henry Hudson; Swan River, formerly Black Swan River, where someone saw such a bird for the first time) or descriptively (Little Twisty Green River). "Mississippi River" (among other native names) is anomalous in this view, because the name belonged to the river first; I can only suppose that it was absorbed into the now predominant pattern, perhaps after a State was named after it.

In a phrase like "the river Thames", the R ought not to be capitalized in my humble opinion.