I'd like to note that you can also see 13.6 billion light years away with your eyes. There's no limit to distance other than whether or not there's anything to see. Telescopes collect light; they don't somehow probe out to a certain point and stop there.

The functional limit is determined by two things: resolution and wavelength. For complicated reasons involving the expansion of space-time, anything really far away is going to show up as longer wavelengths than if you were close by it. The longer the wavelength, the bigger a detector you need to gain good resolution. JWST looks at part of the infra-red range, so it's calibrated generally for further objects than (for instance) Hubble was, and consequently the collection area (the mirrors) had to be a lot bigger. That imposes a limit of how detailed an image we can resolve.

Resolution itself is a bit more straightforward: sure, you can see far away, but how much detail can you see? Not much! Telescopes like this can see with many times the resolution of a human eye, but they're also looking at things that would be unimaginably tiny in the sky. Remember, if something is twice as far away, it will appear half as wide. And now we're talking about things many of orders of magnitude further away than anything you can make out in detail with your eye. That's the limit that's important here, not strictly how far away it is.

If you look at the moon with your eyes, you can see a decent amount of detail. You can see some different kinds of terrain. You probably can't see the small craters that cover its surface. They just don't take up enough angular size in your vision.

The sun, coincidently, is about the same width in our sky. Yet it is 400 times wider than the moon. It's also about 400 times further away.

Alpha Centauri, the sun's nearby star system, is about 100 million times further away from us than the moon is. To see one of the stars there in the same detail we can see from the nearby moon, we'd need a telescope with 100 million times the resolving power of our eyes. And we just don't have one. Not even Webb comes close. Webb has an angular resolution of about 0.1 arc seconds. The human eye can do about 60 arc seconds, so Webb's acuity is about 600 times greater than the human eye. That's great, but a very far shot from 100 million. And we're talking about something the size of a star, and one of the very nearest ones. Looking at a planet much further away...it's simply way out of our capability.

The cool thing though is that there are tricks we can use to add different observations together into higher quality ones, so we aren't as far away from the goal as it looks. But it's still very complicated and takes a lot of combined observing time, and you still need to target to be bright enough to detect well, which distant planets aren't.