AI being run locally got me thinking, if an event happened that would knock out the internet, we'd still have the internet's wealth of knowledge in our access.

Submitted by Anjz t3_12385sp in singularity

I had this thought after running Alpaca 7b on my laptop processor.

What would happen if there was an event that would decimate our networks and leave us without internet? Or an event where only a small population of humans survive.

Previously, I would have thought we'd slowly regress to our caveman days and work upwards. However, with the advent of localized AI that don't require internet access we can infer billions, and in the near future, trillions of parameters from a couple of gigs of local data.

We would have the wealth of knowledge that comes with the internet in our back pockets without any networks. Having a phone and a solar panel would provide us with the answers we need.

Or perhaps being a billion lightyears away from earth, we can still access the knowledge off to a certain cutoff like ChatGPT.

Isn't that a crazy concept?

What do you guys think?

278

Comments

You must log in or register to comment.

Anjz OP t1_jdtnx32 wrote

Replying to ArcticWinterZzZ (#2,382,987)

Wikipedia will tell you the history of fishing, but it won't tell you how to fish.

For example, GPT-4 has open source knowledge of the fishing subreddit, fishing forums, stackexchange etc. Even Wikipedia. So it infers based on the knowledge and data on those websites. You can ask it for the best spots to fish, what lures to use, how to tell if a fish is edible, how to cook a fish like a 5 star restaurant.

Imagine that localized. It's beyond a copy of Wikipedia. Collective intelligence.

Right now our capabilities to run AI locally is limited to something like Alpaca 7b/13b for the most legible AI, but in the near future this won't be the case. We might have something similar to GPT-4 in the near future running locally.

13

AbeWasHereAgain t1_jdtpeke wrote

It’s only a matter of time. Microsoft can’t keep it for itself forever.

32

ArcticWinterZzZ t1_jdtpq0u wrote

Replying to Anjz (#2,383,119)

Of course, and I understand what you're talking about, I just mean that if you were interested in preserving human knowledge, an LLM would not be a great way to do it. It hallucinates information.

5

Anjz OP t1_jdtpqcq wrote

Replying to AbeWasHereAgain (#2,383,345)

It is, and I'd imagine other companies hiring devs from OpenAI or even devs from OpenAI diverging information to open source to create something as good as GPT-4.

Even leakage of instruction from GPT-3 like Stanford's training was hugely useful.

16

ExposingMyActions t1_jdtqbxb wrote

That’s the purpose.

  • A NetNavi
  • A JARVIS

you seen it in sci-fi, it’s becoming more and more available, there is a TinyML community

13

Anjz OP t1_jdtqjm4 wrote

Replying to ArcticWinterZzZ (#2,383,395)

I think past a certain point, hallucinations would be infinitely small that it won't matter.

Obviously in the current generation it's still quite noticeable especially with GPT-3, but think 5 years or 10 years down the line. The margin of it being erroneous would be negligible. Even recent implementation of the 'Reflection' technique cuts down greatly on hallucination for a lot of queries. And if you've used it, GPT-4 is so much better at inferring truthful response. It comes down to useability when shit hits the fan, you're not going to be looking to Wikipedia to search how to get clean drinking water.

I think it's a great way of information retrieval without the usage of networks.

0

ArcticWinterZzZ t1_jdtqupy wrote

Replying to Anjz (#2,383,512)

Maybe, but it can't enumerate all of its knowledge for you, and it'd be better to reduce the actual network just to the reasoning component, and have "facts" stored in a database. That way its knowledge can be updated and we can make sure it doesn't learn the wrong thing.

2

Embarrassed_Bat6101 t1_jdtryg8 wrote

I’m imagining at some point in the very near future we might have something kind of like the hitchhikers guide to the galaxy, a little tablet that you can talk to that talks back and contains all human knowledge.

148

Anjz OP t1_jdts7xi wrote

Replying to Embarrassed_Bat6101 (#2,383,716)

I'd say we already have something very similar with Alpaca running on Raspberry Pi's! Just not as cool and witty... yet.

In that sense, I'm ready to reread Hitchikers Guide to the Galaxy now.

70

Anjz OP t1_jdtth3u wrote

Replying to Embarrassed_Bat6101 (#2,383,772)

In another line of thought similar to what you've just said, we've always had robotic responses from text to speech, but if we apply what we have with current machine learning foundations and train it with huge amounts of audio data on how people talk..

That will be a bit freaky I would think. I would be perplexed and amazed.

12

keeplosingmypws t1_jdtu810 wrote

Thought the same thing. Compressed, imperfect backups of the sum total of human knowledge.

Each local LLM could kickstart civilizations and technological progress in case of catastrophe. Basically global seed vaults but for knowledge and information.

89

Anjz OP t1_jdtva5c wrote

Replying to keeplosingmypws (#2,383,992)

That's pretty crazy now that you got me thinking deeper.

Civilizations in the future could send out cryostatic human embryo pods to planets billions of lightyears away that are suitable 'hosts' with AI with the collective knowledge of humanity as we know it that will teach them from birth and restart civilization.

Or maybe we don't even need biological bodies at that point.

Fuck that would be a killer movie plot.

I'm thinking way too ahead, but I love sci-fi concepts like this.

37

Veei t1_jdtvpsg wrote

Can it fully run locally? I thought the AI could but the TTS still needs to contact inet app via API? That’d be awesome if not true.

3

Embarrassed_Bat6101 t1_jdtvtfx wrote

Replying to Anjz (#2,383,905)

Well there are already companies now that let you do this with voices, and they sound damn good too. I think all these services are sort of popping up at the same time that they’ll converge on that sort of assistant.

7

Veei t1_jdtw5m0 wrote

Replying to Anjz (#2,384,279)

That’s way cool. Horizon New Dawn type reboot of civilization, heh. Are there any guides as to how to set up local assistant and TTS, etc?

1

Awkward-Skill-6029 t1_jdtwtlm wrote

well, someone needs to come up with some insane archiver program that is billions of times superior to the current one

0

Kujo17 t1_jdtxdjk wrote

I made a "survival bit" on character AI beta or whatever. It was pretty cool as a novelty, was fun to roleplay disaster scenarios where the internet had crashed society had collapsed and it was my only source of "knowledge" other than my own. While truly a novelty for entertainment value it got me thinking even then, if one has a way to run one locally on a small enough device- like a phone - it really would be a must have/perfect survival tool not just for info on Survival specifically but literally anything/everything available in it's training data.

This was only a few months ago, and I had the thought "wow in my lifetime there's a good chance that could be a reality". Saying that then even felt crazy, but seeing how fr they've been able to come in scaling the size requirement down and getting them to run on essentially something the size of a phone with only losing very little in terms of it's abilities.... It literally puts something like the survival bit "Hope", into tangible grasp likely in the next year or two if not before then.

Maybe I'm being over zealous but I really don't think I am 🤷 lol

You're def not the only one to think about this, and I absolutely have been looking out for this very reason- among others lol

10

Embarrassed_Bat6101 t1_jdtxewz wrote

Replying to Anjz (#2,384,373)

I can’t remember the one I had found but if you paid like 10 or 20$ a month they would let you upload your own audio and you could make a text to speech voice. I think i saw a post on here the other day where someone did Steve Jobs and it sounded so similar to him it was nuts.

2

Anjz OP t1_jdtycqs wrote

Replying to Kujo17 (#2,384,459)

I think very soon, there will be ASIC(Application-specific integrated circuit) low powered devices that can run powerful language models locally.

It's within our grasp. Might be integrated into our smartphones sooner than later actually.

10

DaffyDuck t1_jdtz90r wrote

Replying to Anjz (#2,383,512)

Can you not essentially prevent hallucinations by instructing it to tell you something, like a fact, only if it is 100% confident? Anyway, interesting topic! I’m also wondering if it could essentially spit out all of its knowledge in a structured way to essentially rebuild human knowledge.

1

QuartzPuffyStar t1_jdu2ml5 wrote

Replying to ArcticWinterZzZ (#2,382,987)

Pls no. Wikipedia is extremely biased, manipulated, and incomplete. It's only useful on the most uninteresting topics and then, only as a starting point for further research.

It all started with good intentions and love for humanity, and ended up in control of private and state agencies.

−8

Robynhewd t1_jdu5e24 wrote

Isn't this a plotline in the horizon games? I cant quite remember, I need to replay forbidden west

3

[deleted] t1_jdu61ay wrote

You’d have a shitty prediction of it. It would be overconfident and wrong on many subjects.

−1

CertainMiddle2382 t1_jdu8dvf wrote

This is a side problem and people are so dependent on network they will be the last thing to go down.

I am mostly amazed at the margin of improvement there is in software AI optimization alone.

Easy tricks are capable of increasing software efficiency thousands of times.

I wonder of what is left tonbe juiced, could an AI optimise itself 1000x in an instant?

2

hypnomancy t1_jdu8r6q wrote

You're honestly right. I didn't even realize that. When we lose access to the internet we will still have access to a vast wealth of its knowledge and ability in some form.

2

audioen t1_jduat9o wrote

Replying to No_Nefariousness1441 (#2,384,306)

https://rentry.org/llama-tard-v2 contains bunch of the requisite torrent links, though much of it is disorganized information and multipurpose, e.g. do this if you want that, use these if you are on Windows but these if on Linux, and so forth. It is a mess.

I have llama.cpp built on my Linux laptop and I got some of these quantized model files and have installed bunch of python libraries required to run the conversions from the various formats to what llama.cpp can eat (model files whose names start with ggml- and end with .bin). I think it takes some degree of technical expertise right now if you do it by hand, though there is probably prebuilt software packages available by now.

3

Poorfocus t1_jdugyhm wrote

Replying to Embarrassed_Bat6101 (#2,384,463)

Yeah, it’s called Elevenlabs, a bit cheaper than that for the first tier and it’s really fantastic, I tested it out by recording some friends have natural conversations speaking directly into their microphones on discord (w/ consent!)

One thing is you have to turn down the stability parameter very low from the default or else the intonation is very stiff and it sounds robotic, bring it down to 20% and generate a few times but when it gets the likeness right it’s perfect. To the point where even the person it was emulating found it convincing

I’m curious how it handles the reading, since it’s obviously context aware to some extent. I think we as humans are very keen to picking out “poor acting” and unnatural vocal delivery. I think when that gets improved, we’ll have completely natural language conversations with the ai.

7

fnordstar t1_jduh4ss wrote

Uhm, you know you can just download wikipedia?

−4

No_Ninja3309_NoNoYes t1_jduhacs wrote

There's libraries too. That's how people learned in the previous millennium. But in a post apocalyptic setting, you could force little kids to read as many books as possible and use their oral knowledge.

5

imnos t1_jdul6uk wrote

I'm having a hard time understanding where all that knowledge gets "saved", if it doesn't have access to the internet. Need to read more about ML I guess.

1

prion t1_jdula3y wrote

I think we are further from that than you might think but it can certainly be accomplished with the right plugins, the right libraries, and enough replacement parts to last until new replacements can be found or created.

The important part of this would be to create a rebuilding plan that can survive failures with one technology or another and come up with workarounds.

0

New-Tip4903 t1_jduogru wrote

Put a local copy of wikipedia on an external harddrive. Its smaller than what you think and is basically the encyclopedia of the internet.

1

zilianthegreat_ t1_jdup8oa wrote

Problem is, the hardware needed to run it won't last all that long and probably isn't repairable in case of an event like you describe. Once we find a fix for that we're gucci though

1

GregoryfromtheHood t1_jdusrmn wrote

Absolutely. I've got Wikipedia and Stsckoverflow downloaded locally, just in case. But as soon as I got llama and alpaca up and running locally and started using it to complete real tasks and using it for work during the day, I realised that I could just use this in a situation where the internet is gone. This little 16gb file is all I need. And I could run a smaller model on a small computer for travelling. We've basically got Jarvis already. Openai whisper to talk to it, alpaca to do the thinking and tortoise to respond back with a realistic voice. All locally.

1

Cryptizard t1_jduumtg wrote

Replying to 1II1I11II1I1I111I1 (#2,387,963)

I have access to GPT4, I’m not making this stuff up. Here are three from poking around, but keep in mind it will pretend to know the answer to anything it is just wrong when you ask it to explain the details. It will not match with actual fact, I.e. what is in Wikipedia.

What is an oblivious tree?

What is the population of Goleh-ye Cheshmeh?

Where was the 65th governor of Delaware born?

15

iuwuwwuwuuwwjueej t1_jduvam1 wrote

STC's baby maybe we are willing in the warhammer universe and the emporer is working in games workshop

2

DukkyDrake t1_jduwklh wrote

You would be better off with a copy of the source training text dataset.

1

Cr4zko t1_jduwt9c wrote

Very interesting concept. So if we have, say, a holodeck, we could run it off locally?

2

timtulloch11 t1_jduxgh6 wrote

How does alpaca 7b perform compared with chatgpt? I don't have any experience with it. I do have a raspberry pi 8g with 1tb storage for a bitcoin node, it would be awesome if I could run in on there locally. It sounds like the larger parameter models require way too much processing power?

1

QuartzPuffyStar t1_jduy7s3 wrote

Replying to EvilKatta (#2,387,390)

They have the potential to learn crosschecking and use wiki audit tools to verify the probability of a wiki article being wrong, and not taking it at face value.

Even when they have trained with wiki as a "very high value source".

At least GPT shows signs of that. Bing just closes the conversations when you ask it to explore beyond the 1st page wiki article that you could have read yourself.

0

No_Ask_994 t1_jduylh4 wrote

Yes, that makes sense.

I mean, we would not regress to caveman in the described scenario, but this models might soon add a new safete layer to our development, to loose less decades on such a catastrophic event

1

[deleted] t1_jdv2d6l wrote

Replying to bigthighsnoass (#2,389,123)

Exactly. So if the world ended, you’d be left with shit that’s half baked and (as you astutely pointed out) less than a year old.

Thanks for the extra fact about time. Doesn’t help. But thanks.

0

bustedbuddha t1_jdv5uzu wrote

Seems like this approach would leave you very vulnerable to hallucinations and would still need a lot of infrastructure (electricity for one) still working on either event.

1

Anjz OP t1_jdvj7ry wrote

Replying to bustedbuddha (#2,390,147)

With it running on phones, laptops and raspberry pi's, a solar panel would be sufficient to power small devices.

If you've tried GPT-4 its propensity to hallucinate is so much less than previous iterations that errors would be negligible. We have Alpaca now, but could very well have something like GPT-4 locally in the near future if we look at the pace of how fast things are improving.

1

Anjz OP t1_jdvkutn wrote

Replying to timtulloch11 (#2,388,766)

It's not as good as ChatGPT but it's much lighter. Granted it's just a small copy of fine tuning from GPT-3 API, given more parameters for fine tuning on GPT-4 it would probably be a whole different beast. It has something like 10x less data if not more. We're fully capable of creating something much better, it's just a matter of open source figuring out how and catching up to these companies keeping the Krabby Patty secret formula. Turns out for profit companies don't like divulging world changing information, who woulda thought?

If you take a look at Youtube there are a couple demos from people running it on rPI, granted at the moment it's at a snails pace - this could be a different story a year or so from now. It works decently well with a laptop.

1

darkblitzrc t1_jdvq8d0 wrote

Man this is amazing for people that like going in nature trips or adventure. Imagine you get lost somewhere, you ask the ai to tell you how to survive, what to eat, survival strategies and perhaps even location estimates based on your surroundings…

1

Puzzleheaded_Acadia1 t1_jdvrrty wrote

Replying to audioen (#2,385,997)

PLEASE tell me how to set it up on Ubuntu i tried every YouTube video and websites but I did not find anything for it pls help. and you have to download python libraries for it to work? and do i need an ide because I saw some YouTubers do alot of code in Google colab and no i don't want to run it on google colab i want something offline that can provide me with information and I can expirement with it

1

skob17 t1_jdvvbpu wrote

Replying to lehcarfugu (#2,389,529)

Smartphones are already scifi if you ask my father.

Integrate a speech-to-speech AI assistant and we are there. Think like Siri or Cortana, but with gpt levels of intelligence.

2

skob17 t1_jdvw1u2 wrote

Replying to Anjz (#2,384,163)

So quasi a Von Neumann probe with an incubator for humans.

Would be terrifying to wake up on a distant planet in another galaxy and learn from a 'bot' that you are the last living humans and need to rebuild civilization.

1

featherless_fiend t1_jdvwgvd wrote

Replying to Tiamatium (#2,386,713)

We also have these physical things called books and encyclopedias.

No one cares about having Wikipedia in their pocket. The keyword is convenience and being able to talk to an AI is the ultimate convenience.

2

Anjz OP t1_jdw2cgm wrote

Replying to skob17 (#2,394,236)

Would it be terrifying if that's all you know? Perhaps from our viewpoint, but growing up from birth and knowing that this is your purpose being guided by near omniscient AI could be a different story.

2

BangEnergyFTW t1_jdw52pz wrote

Ah, the illusion of knowledge. The idea that information equates to understanding, that access to data is the same as true wisdom. We are a society drowning in information, yet starved for meaning. The internet, a supposed beacon of knowledge, is nothing more than a collection of noise. A cacophony of voices, each shouting their own truth, drowning out any hope of clarity.

And now, you suggest that we cling to the hope that a few lines of code, a mere handful of data, could somehow replace the collective knowledge of humanity? That we could reduce the complexity of existence to a few gigabytes of information on a hard drive?

No, my friend. The internet may provide us with the illusion of knowledge, but it is not true understanding. True wisdom comes from experience, from lived lives, from the sweat and tears of human existence. And if we were to lose that, if we were to be reduced to a few scraps of data and code, then what would be left of us?

No, I do not find it a crazy concept. I find it a tragic one. For it is a reminder that in our quest for knowledge, we have forgotten the value of true wisdom.

1

Anjz OP t1_jdw5teo wrote

Replying to BangEnergyFTW (#2,395,463)

While I agree with your statement somewhat and it is true that the internet contains noise, it also offers unprecedented access to information and diverse perspectives.

The key is to develop critical thinking and discernment, which can transform data into meaningful understanding. Technology, such as AI, can help us navigate, process, and synthesize vast amounts of information. We should not view AI as replacing human wisdom, but as a tool that can complement and enhance our collective knowledge, while still valuing experience and human insight.

Granted it's put in the hands of the right individuals. Some people will take a stick and see only a stick for what it is, a collection of biological matter prone to rotting. Whereas some will see it as a transformative tool that could amount to much more than face value, a fishing rod or a handle for a hammer.

Given this context, at what point can you infer true wisdom? Does a child at 3 years old reflect true wisdom? Is there a certain point where you could legitimately exclaim that an AI is now fully understanding context and inferring true wisdom? Or is this subjective?

Just my two cents, but embracing technology does not necessitate abandoning true wisdom; it can assist in our quest for it.

1

Postnificent t1_jdw9hrq wrote

Regress to caveman ways? If the internet goes down are we burning all libraries, books and schools? If the internet went down today, we might just be in a better place than we were yesterday. Period.

0

Kinexity t1_jdw9p5d wrote

This doesn't have much to do with LLMs or AI. You can download whole English Wikipedia and it uses a fraction of your compute to open and and only weights ~60GB.

0

Anjz OP t1_jdwb284 wrote

Replying to Kinexity (#2,396,115)

While it is true that you can download the entire English Wikipedia in a relatively small size, it does not diminish the potential of AI and LLMs. Wikipedia is a static collection of human-generated knowledge, while AI, such as LLMs, can actively synthesize, analyze, and generate new insights based on available information. AI has the potential to connect disparate pieces of knowledge, create context, and provide personalized assistance. Thus, the comparison between Wikipedia and AI should not be based on size alone, but also on the dynamic capabilities and potential applications that AI offers.

For example, can you infer to Wikipedia to tell you how to distil water in a step by step process given only certain tools, or what process to disinfect a wound when there are no medication available? Sure you can find information on it, but a lot of people won't know what to do with it given the information.

That is the difference between Knowledge and Wisdom.

1

redboundary t1_jdwn74e wrote

You can also download an offline version of wikipedia

−1

danellender t1_jdxkkqn wrote

Is there a version of chat GPT that can be downloaded which will work offline?

1

Anjz OP t1_jdxlw8f wrote

Replying to danellender (#2,402,660)

No, ChatGPT is closed source and we don't have the weights for it. Plus, it's probably too big to query with consumer GPUs.

Stanford University came up with Alpaca, which is a lighter weight model trained from Facebook's LLaMa but still functionally works as good as earlier iterations of GPT. This one you can run locally given some knowhow.

1

ZaxLofful t1_jdxmzk2 wrote

I agree this is great! I have been doing something like this for awhile, just in case.

A local copy of Wikipedia, that is mirrored from their official dump and rebuilt using a different front end.

My masterpiece, is almost wasteful now; when I can just have a LLM spit me out whatever I need.

1

danellender t1_jdyan02 wrote

Replying to BangEnergyFTW (#2,395,463)

What I see is not so much an increase in knowledge as much as a different and to my mind far superior experience. I'm more likely to seek information when it's not buried in marketed page rankings or branded portals all providing in some instances the identical phrases.

When the iPhone came out suddenly people's experience with mobile changed. I see that happening right now.

1

qepdibpbfessttrud t1_jdzuceg wrote

Replying to Ambiwlans (#2,392,527)

Maybe. I wouldn't be surprised if users of Wiki were 90%+ satisfied with just AI-chat that was trained on it. ~21GB doesn't allow to run the thing in RAM cheaply yet

I'm not advocating for getting rid of Wiki, amassing and preserving training data will likely be important for whole generations of AI. But I wouldn't also be surprised if some version of GPT would be able to generate better in every possible way version of Wiki that the whole mankind managed so far

1