Argthtjtr

There are many questions on WBSE that cover most aspects of building a planetary system. (some examples below):

Creating a realistic world(s) map - planetary systems

How many planets should I have in my planetary system?

What distances would be involved in this planetary system?

But it occurred to me that while a sci-fi story setting might need the complete details of the system, a story set in a medieval or any similar society with pre-telescope or primitive telescope technology would only need to build the parts of the system that would be observable by the inhabitants of participants of the story, or the well known and widespread knowledge of the society that those participants are members of.

So ... How far away (maximum distance) can humans detect planets with the naked eye?

Assume best observable conditions for naturally occurring planets. As far as I can tell from my limited research, this should limit the albedo of the observed planet to about .8 (unless you can give a reasonable explanation of why it should be more or less than that), and the radius of the observed planet should be no more than about that of Jupiter (again, unless a reasonable reason for an exception is given). Assume ideal observing conditions, such as no light pollution, good (perfect specimen human) eyesight, ideal alignment of star and observing planet and observed planet for best lighting of observed planet, etc. Also assume a sun-like star, and earth-like planet as far as atmosphere and other observation characteristics, though human habitability is not a requirement except as it directly relates to human-like observation (a fictional atmosphere is allowed, as long as there is an explanation of how it improves observation, while not entirely preventing complex life in general). Earth-like orbit is NOT required for either planet.

This question has applications not only for general world-building, but can also be used as a basis for calculating orbit times which then apply to things like creating mythologoes, calendars, religious influences, cultural iconography, and much more.

With the naked eye, humans can see approximately 6th-magnitude objects. We can compute the apparent magnitude of a planet at a given distance, and find the distance corresponding to an apparent magnitude of +6. The formula is
$$m_p=M_p+5logleft(frac{d}{10text{ pc}}right)$$
where $m_p$ and $M_p$ are the apparent and absolute magnitudes and $d$ is the distance from Earth to the planet. $M_p$ can be calculated if we know $M_S$, the absolute magnitude of the Sun:
$$M_p=M_S-2.5logleft(afrac{R_p^2}{4d_s^2}right)$$
where $d_s$ is the distance from the planet to the Sun and $r_p$ is its radius. Let's say that $d_sapprox d$, by assuming that the planet is much further from the Sun than Earth is. Finally, we get
$$m_p=M_S-2.5logleft(afrac{R_p^2}{4d^2}right)+5logleft(frac{d}{10text{ pc}}right)$$
This has the solution
$$d=sqrt{frac{a^{1/2}R_Jcdot10text{ pc}}{2}}10^{cfrac{m_p-M_S}{10}}$$
Let's say $R_papprox R_J$, the radius of Jupiter, and $a=0.5$ - also like Jupiter. Then, given that $M_S=4.83$, I get 24 astronomical units - about halfway between Uranus and Neptune. We can't see Neptune with our naked eyes, but we can see Uranus under good conditions, which matches our calculations (although note that the ice giants have different albedos). Let's say we choose an even higher albedo - say, $a=0.8$, as you suggested. This gets me up to 27, even closer to Neptune.

Uranus was detected by naked eye in the classical age but because it is too slow it wasn't recoginzed as a planet. In the "discovery" section. So the classical astrological manuals that are aware of its existence classify the planet as a fixed star.

So it's not only visibility that matters but also the speed because if it is too slow it will be classified as fixed star.