Submitted by timscarfe t3_yq06d5 in MachineLearning
Nameless1995 t1_ivpyc47 wrote
Reply to comment by PrivateFrank in [D] What does it mean for an AI to understand? (Chinese Room Argument) - MLST Video by timscarfe
> Intentionality and understanding and first-person (phenomonologistic) concepts, and I think that's enough to have the discussion. We know what it is like to understand something or have intentionality. Intentionality in particular is a word made up to capture a flavour of first-person experience of having thoughts which are about something.
> I think that to have "understanding" absolutely requires phenomenal consciousness. Or the "understanding" in an AI has could be the same as how much a piece of paper understands the words written upon it. At the same time, none of the ink on that page is about anything - it just is. There's no intentionality there.
That's exactly Searle's bigger point with CRA. CRA is basically in the same line as Chinese Nation and Dneprov's game, paper simulations, and such arguments (not particularly novel). The larger point of Searle is that although machines may understand (according to Searle, we are such machines), programs don't understand by simply the characteristics of the programs. That is particular instantiation of programs (and we may be such instantiation) may understand depending on the underlying causal powers and connections instantiating the program but not any arbitrary instantiation may understanding (for example "write symbols in paper" or "arrange stone" instantiations).
The stress is exactly that in any arbitrary instantiation may not have relevant "intentionality" (the point becomes muddied because of the focus on the simplicity of the CRA.)
However, a couple of points:
(1) First I think at the beginning of disagreements, it's important to set up the semantics. Personally, I think a distributionalist attitude towards language. First, I am relatively externalist about meaning of words (meanings of words are not decided by solo people in the mind, but it's grounded in how the word is used in public). Second, like Quine, I don't think most words have clean determinate "meaning" and especially more so for words like "understanding". This is because there are divergences among how different individuals use words, and there may be no consistent unfying "necessary and sufficient" rules explaining the use of words (because some usages by different people may contradict each other). Such issues don't express that much in day to day discourse, but becomes more touchy when it comes to philosophy. So what do we do? I think the answer is something like "conceptual engineering". We can try to make suggestions about refinements and further specifications of nebulous concepts like "understanding" when it's necessary for some debate. Then it can be upto the community as a whole to accept and normalize those usages in necessary contexts or counter-suggests alternatives.
(2) With the background set up in (1), I believe "understanding"'s meaning is indeterminate. Now from a conceptual engineering perspective, I feel like we can go multiple different mutually exclusive branches in trying to refine the concept of "understanding". However, we need to have some constraints as well (we need to ideally keep the word's usage still roughly similar to how it is).
(3) One way to make the notion of "understanding" more determinate, is to simply stipulate the need of "intentionality" and "phenomenality" for understanding.
(4) I think making intentionality essential for understanding is fair (I will come back to this point), but not sure if "phenomenality" is as essential for understanding (not needed for the word to roughly correspond to how we use it).
(4.1) First note that it is widely accepted in Phil. of Mind, that intentionality doesn't require phenomenality. There is also a debate on whether intentionality ever has a phenomenal component at all. For example, Tye may argue in a phenomenal field, there is just sensations, sound, images, imageries etc. no "thoughts" representing things as about something. When thinking about "1+1=2", Tye may say the thought itself is not phenomenally present, instead you would have just some phenomenal sensory experiences associated with the thought (perhaps your a vague audio experience of 1+1=2, perhaps some visual experiences of 1,+1,=2 symbols in imaginations and so on). Functionalists can have a fully externalist account of intentionality. They may say say that some representation in mental state being "about" some object in the world is simply a matter of having the relevant causal connection (which can be acheived by any arbitrary instantiations of a program with proper interface to the io signals - "embedding it in the world") or relevant evolutionary history (eg. teleosemantics) behind the reasons for the selection of the representation-producer-cosumer mechanism. This leads to the "so-called" causal theories of intentionality. They would probably reject intentionality as being some "internal" flavor of 1st person experience, rather than it being grounded in the embodiment of the agent.
(4.2) Even noting Anscombe's work on intentional theories of perception that was kind of one of the starting points on intentional theories -- she was pretty neutral on the metaphysics of intentionality and she was trying to take a very anti-reificationist stance, even close to treating it more of a linguistic device. She also distinguished intentional content from being mental objects. For example, if someone's worshiping has the intentional object -Zeus (they worship Zeus), then it's wrong to say that the intentional content is "the idea of zeus" (because it's wrong to say that the subject is simply worshipping the idea of Zeus, because the subject's 'intention' is to worship the true Zeus which happens to not exist - but that's irrelevant). This raises the question - what kind of metaphysical states can even constitute or ground this "subject taking the intentional content of her worship as being Zeus" (the intentional content is not exactly the idea of Zeus, or Zeus-imageries that the subject may experience -- but then what does this "intentionality" correspond to?). After thinking I couldn't really think of any sensible answer besides again going back to functionalism: the intentional object of the subject is zeus because that's the best way to describe their behaviors and functional dispositions.
(4.3) That's not the only perspectives of intentionality of course. There are for example, works by Brentaro and approaches of intentionality from core phenomenological perspectives. But it's not entirely clear if there is some sort of differentiable "phenomenal intentionality" in phenomenality. I don't know if I distinctly experience "aboutness" rather than simply having a tendencing to use "object-oriented language" in my thinking (which itself isn't distinctly or obvious phenomenal in nature). Moreover, while me understanding certain concepts may "feel" phenomenally to be a certain way, it feels to be a very poor account of understanding. Understanding is not a "feeling" nor it is clear why having the "phenomenal sense" of "understanding xyz" is necessary. Instead upon reflection of me for example having the "understanding of arithmetic" constitutes, I don't find it to be necessarily associated with my qualitatively feeling a certain way with dispositions to say or think about numbers, +, - (they happen, and the "qualitative feeling" may serve as a "mark" signifying understanding -- "signifying" in a functional correlational sense), but it seems to be most meaningfully constituted by possession of "skills" (ability to think arithmetics, solve arithmetical problems etc.). This again leads to functionalism. If I try to think beyond that, I find nothing determinate in 1st person experience constituting "understanding".
Nameless1995 t1_ivpyckd wrote
(5) Given that I cannot really get a positive conception of understanding beyond possessing relevant skills, I don't have a problem in broadening the notion of understanding to abstract out phenomenality (which may play a contingent (not necessary) role in realizing understanding through "phenomenal powers"). So on that note, I have no problem with allowing understanding (or at least a little bit of it) to a "system of paper + dumb-rule-follower + rules as a whole" producing empty symbols in response to other empty symbols in a systematic manner such that the it is possible to map the input and output in a manner making it possible to interpret it as a function to do arithmatic. You may say that these symbols are "meaningless" and empty. However, I don't think "meaning" even exists beyond just functional interactions. "meaning" as we use the term simply serves as another symbol to simplify communication of complicated phenomena wherein the communication itself is just causal back-and-forth. Even qualia to me are just "empty symbols" gaining meaning not intrinsically but from their functional properties in grounding reactions to changes in world states.
(6) Note I said "little bit of it" regarding understanding of arithmetic the system "paper + dumb-rule-follower+ some book of rules" as whole. This is because I am taking understanding as a matter of degree. The degree increases with increase in relevant skills (for example, if the system can talk (or having functional characteristics mappable to) about advanced number theory, talk about complex philosophical topics about metaphysics of numbers, then I would count that as "deeper understanding of arithmatic")
(7) 5/6 can be counter-intuitive. But the challenge here is to find an interesting positive feature of understanding that the system lacks. We can probably decide on some functional characteristics or some need of low-level instantiation details (beyond high level simulation of computational formal relations) if we want (I personally don't care either way) to restict paper+dumb-rule-followers to simulate super-intelligence. But phenomenality doesn't seem too interesting to me, and even intentionality is a bit nebulous (and controversial; I also relate to intentional-talk being simply a stance that we take to talk about experiences and thoughts rather than taking it as something metaphysically intrinsic in phenomenal feels (https://ase.tufts.edu/cogstud/dennett/papers/intentionalsystems.pdf)). Some weaker notion of intentionality can definitely be allowed already in any system behavior (including paper-based TM simulation as long as that is connected to a world for input-output signals). Part of the counter-intuitive force may come from the fact is that our usage of words and "sense" of a word x applies in context y, can be a bit rooted in internal statistical models (the feeling of "intuition" that it doesn't feel right to say the system "understands" is the feeling of ill-fittingnes due to our internal statistical models). However, if I am correct that the words have no determinate meaning -- it may be too loose or even have contraidcting usages, in our effort to clean them up through conceptual engineering it may be inevitable that some intuition needs to be sacrificed (because our intuitions themselves can be internally inconsistent). Personally, considering both ways, I am happier to bite the bullet here and allow papers+dumb-writers to understanding things as whole when the individual parts don't: simply following from my minimalist definition of understanding revolving around highl level demonstrable skills. I feel like more idealized notion are hard to define and get into mysterian territories while also unnecessarily complicating the word.
Viewing a single comment thread. View all comments