Submitted by No_Draft4778 t3_11rmgzs in MachineLearning
currentscurrents t1_jcfu9l8 wrote
Reply to comment by sam__izdat in Modern language models refute Chomsky’s approach to language [R] by No_Draft4778
That's why it's a natural language instead of a formal language.
WikiSummarizerBot t1_jcfub6d wrote
>In neuropsychology, linguistics, and philosophy of language, a natural language or ordinary language is any language that has evolved naturally in humans through use and repetition without conscious planning or premeditation. Natural languages can take different forms, such as speech or signing. They are distinguished from constructed and formal languages such as those used to program computers or to study logic.
>In logic, mathematics, computer science, and linguistics, a formal language consists of words whose letters are taken from an alphabet and are well-formed according to a specific set of rules. The alphabet of a formal language consists of symbols, letters, or tokens that concatenate into strings of the language. Each string concatenated from symbols of this alphabet is called a word, and the words that belong to a particular formal language are sometimes called well-formed words or well-formed formulas. A formal language is often defined by means of a formal grammar such as a regular grammar or context-free grammar, which consists of its formation rules.
^([ )^(F.A.Q)^( | )^(Opt Out)^( | )^(Opt Out Of Subreddit)^( | )^(GitHub)^( ] Downvote to remove | v1.5)
sam__izdat t1_jch1c32 wrote
I'm familiar with the terms, but saying e.g. "imaginary numbers don't exist because they're called imaginary" is not making a meaningful statement. All you've said is that German is not C++, and we have a funny name for that. And that's definitely one of the fuzzier interactions you can have about this, but I'm not sure how it proves that natural languages (apparently? if I'm reading this right...) lack structure.
currentscurrents t1_jch3nic wrote
So why do you think it is a structured formal thing?
sam__izdat t1_jch4kn0 wrote
It is a "structured thing" because it has concrete definable grammatical rules, shared across essentially every language and dialect, and common features, like an infinite range of expression and recursion. If language didn't have syntactic structure we'd just be yelling signals at each other, instead of doing what we're doing now. There would be nothing for GPT to capture.
currentscurrents t1_jch9ulc wrote
Oh, it is clearly structured. Words and phrases and sentences are all forms of structure and we're using them right now.
What it doesn't have is formal structure; it cannot be fully defined by any set of rules. This is why you can't build a rules-based parser that understands english and have to use an 800GB language model instead.
>shared across essentially every language and dialect
Noam Chomsky thinks this, but the idea of a universal grammar is controversial in modern linguistics.
sam__izdat t1_jchg8nd wrote
I'll leave it to the linguists to debate UG and the specifics of what it does and doesn't mean, but commonalities like some sort of hierarchy, recursion, structure-dependence of rules, etc clearly exist, whatever you want to call them. By shared I just mean there's specific things that human cognitive faculties are set up to do and then other (often computationally simpler) things they clearly don't do. But again, if you're just saying natural languages are not formal languages, I guess that's true by definition. It just sounded to me like you were implying something different.
Necessary-Meringue-1 t1_jcm5x7g wrote
just because it's "natural" does not mean it's unstructured or does not have any logic, can you be any more disingenuous than to rely some etymology-based semantics?
Like programmers invented structure
Viewing a single comment thread. View all comments