Viewing a single comment thread. View all comments

Sigura83 t1_iybfjkd wrote

Reply to comment by FirstOrderCat in Better Language Models Without Massive Compute by Tom_Lilja

Just based on language model progress and Kurzweil's predictions. I could easily be wrong, I'm just a random internet person, but I do have a Google TV and it can understand and search pretty dang well from voice alone. It queues up what I like as if I had a personal DJ. Ten years ago my cell phone couldn't even browse the net! Every ten years sees a new, impressive tech and AI looks like the next wave. Robotics is lagging, but some kitchen robots have started up (Flippy) as have a lot of warehouse robotics. Things are changing faster and faster. (Which makes sense as there are now 8 billion people figuring things out. It seemed like 7 billion yesterday!)

1

FirstOrderCat t1_iybg5rh wrote

> it can understand and search pretty dang well from voice alone.

there is component where some model translates your voice to text, but searching part contains tons of human hand-crafted code.

So, current language models are good for some narrow tasks (translation is the main one), but still not on the level of abstract thinking human posses. My bet they won't be able, unless some large advancement.

1

Sigura83 t1_iybiukl wrote

I was impressed by the Google Say-Can project. I think we'll see an AI model that can do science pretty soon, it's the goal of Deepmind, and they've done amazing things. And, I think Google moved to a Transformer AI for search two years ago...

I did read a recent paper saying LLM were just doing fancy statistics and hardly thinking at all, but if it's sorta doing it now in only a few years it'll surpass Human level.

If I were to bet, I would say an evolutionary algo + interacting LLM based robots will be beyond anything we can imagine. First in a MMO like setting, then the real world. But we'd need super duper computers to do it, which we might not get because Moore's law may be done. Well... hmm... if the AI MMO was text based, the LLMs could run wild and evolve away without costing too much energy

I dunno... just dreaming! ^_^;;;

1

FirstOrderCat t1_iybm4dy wrote

>First in a MMO like setting, then the real world.

nop, this transition is very hard because of following reason:

current wave of AI can approximate giant datasets, that's something it is doing very well. So, all your examples is: they throw terrabites of data on neural network, and it learns patterns. But this kind AI can't generalize and do abstract thinking which means it can't learn from very few examples.

Meaning yes, they can ask AI to play MMO 100 millions times and it will learn from its own mistakes, but you would need to do the same 100 million times in real world, which is not very feasible.

Another issue is that: MMO has much smaller level of freedom than real world, which makes MMO is not a good benchmark.

1