Dall-e 2.5 is looking pretty good actually. Huge step up from the base version

Instruct Pix2Pix demoed something along these lines. Don't think it's public yet, but at least we know it's doable right now

don't get too excited; it's just calling Dall-e 2(.5?) via their api

But how would it know what you want?

Edit: do you mean something like Tiktok where some recommender shows you a bunch of clips (or whatever other generations) it thinks you'd like then optimizes it's "prompts" in the background to your preferences?

I don't understand what you mean by this. Why would people want that?

Nvidia: Hold my beer

I don't think the two are mutually exclusive.

The problem with retrieval though (at least current implementations) is the model can't attend to memory globally the way it does with context memory; you're bottlenecked by the retrieval process having to bring things into context through a local search.

Honestly, I wouldn't be surprised if we're past this hurdle in a matter of weeks:

RWKV showed how you can get an order of magnitude increase in inference speed of LLMs without losing too much performance. How long until someone instruction-tunes their baselines like alpaca did to llama?

the pace of development on these things is frightening.

it already has. You can rerun the training for ~$500, and people have done this and are sharing the weights around.

Here's one: https://github.com/tloen/alpaca-lora

​

On integration with games locally? eh. Still not fast enough yet. I'd give it another year

actually that's not true.

They published their entire codebase with complete instructions for reproducing it as long as you have access to the original llama models (which have leaked), and the dataset (which is open, but has terms of use limitations which is stopping them from publishing the model weights).

Anyone can take their code, rerun it on ~$500 of compute and regenerate the model.

People are already doing this.

Here is one such example: https://github.com/tloen/alpaca-lora (although they add additional tricks to make it even cheaper).

You can download model weights from there and run it in colab yourself.

​

As far as opening their work goes, they've done everything they are legally allowed to do

Ok, so they got what they wanted no? Google stopped "repackaging it and selling it on their behalf". So what's the problem? Why are they complaining again?

context memory is far too low to even consider it right now. ask again in 5 years

Good. Finally. Google made its own chatGPT a year ago and just refused to release it to the public because they believed Chat bots are "not something that people can use reliably on a daily basis,"

I can't help but feel sweet schadenfreude now that they're in a code red because of the contempt these labs treat the public with: oh people aren't ready for these tools so they need to be the moral arbiters to dictate precisely how they are allowed to be used. Like come off it.

I wonder how long they will hold this position when the competition heats up

You're thinking about this all wrong.

Look at some of the projects coming out of Google research to see where this is going:

Phenaki combines LLMs to generate video sequences

Combine that with something like Wordcraft which allows LLMs to retain consistency for much longer form writing (like entire scripts), and you can start to generate long form synthetic videos.

Those on their own are an interesting product already: being able to ask a bot like chatGPT to make you mini movies on any topic that you can edit simply through prompting like chatGPT?

but take it even further: combine that with an algorithm like tiktok's recommender & Invert it as a reward model for RLHF like openAI does with their davinci models and you also then start getting ways for AIs to automatically generate new content that people like, and recommended them to people. A lot of people scoff at tiktok, but it's enough of a scare to Google that YouTube had to copy it.

I think synthetic media can revolutionise how we consume content: Holodecks are not outside the realm of possibility anymore

>12ft has been disabled for this site

>Back then, we all imagined that by the next decade, we would have 16K photorealistic high field of view VR

People are working on this. Companies like Pimax already did 8k (well technically around 6k but whatever) and ~170 FOV (near human level) back in 2020 and they are pushing for ~12k and full human-level FOV (200 degrees) with their next headset.

So funnily enough, yes we are actually still on track to hit 16k by 2026; within a decade of the first gen VR launch.

It's just really expensive (thousands of dollars for headset only, and you need top end thousand dollar GPUs to power it), which relegates it to a small enthusiast market, which is the problem: the top end stuff is not the mass market stuff.

​

Facebook switched their focus from this high-end enthusiast market to go for mass market appeal (because their business model relies on getting as big a user base as possible) which ultimately means commoditizing the tech that was ultra-high end yesteryear.

This is what's making it look like the tech has stalled: but if you think about it: their 2020 headset was basically using smartphone hardware to power what needed a top-end gaming PC in 2016, and at a fraction of the price. That's progress.

​

------------------------

​

As for AR, that's also coming. The difficulty with AR is that the way we were going about it needs a quantum leap in display tech to solve being able to display things at variable focal lengths.

Companies like Magic Leap and Microsoft with hololens explored the limits of what we could do given the limitations of current screens. And they were awful.

So research groups like CREAL are now working on this next generation for varifocal AR, while companies like Facebook and Varjo are going the opposite way with scanning the real world in real time and rendering it on screens. Jury's still out on which would work best, but either way it's progress.

​

Point is, there's a lot of progress going on in the VR industry right now. It's just scattered and most of it isn't mass market.

exactly, yes

These are all from the sub-field of reinforcement learning (RL). The first 2 and the last are evolutionary methods, and from the abstract the third is some flavor of model-based RL.

Any course on reinforcement learning will give you a decent background in understanding the basics of how these classes of algorithms work.

Here's a link to a curated set of resources for beginners/intermediates in RL: https://github.com/andyljones/reinforcement-learning-discord-wiki/wiki

​

I think it'd important to understand the background so you aren't lost when you're trying to apply these on problems, but that said, I won't recommend trying to implement these from scratch yourself: start from open-source baselines instead; there's a lot of tiny details to these algorithms that are hard to test, and one thing going wrong can make the whole algo fail in ways that are really hard to debug.

yeah but python is still a giant pain in the ass and gets in your way the second you try to do anything beyond the standard programming model.

​

That's why there's so much effort these days to make domain-specific-languages inside of python that can compile down to C (tools like Numba, TorchScript, Jax etc).

Somewhere along the line we realized the hidden costs of standardizing around python, but now there's so much infra already invested in it that switching away is impractical. And what can you even switch to?

World's biggest yes, but as recent LLMs and diffusion models show it's not all about size, it's show you use it

I don't know who Dennis is. Do you mean Demis Hassabis of deepmind? If so, sure; they've certainly done great work. DeepMind in paricular has demonstrated their commitment to advancing science and making their progress work for everyone; I'm inclined to believe it's more than lip service when they talk about making the world better.

My only concern with those two (Deepmind and openai) is how much control do these founders really have over the end products? Deepmind in particular has for years been trying to negotiate with google to make themselves operate more like a non-profit for this exact reason that they don't want powerful AI they create to be controlled by 1 for-profit company, but Google declined (I speculate that it's because the whole reason they've invested billions into deepmind is to make a profit off AGI). So yeah, Demis may have the best intentions and honestly mean it, but how much power does he really have to say no to Google?

​

Altman & OpenAI was having similar funding issues a few years back; they were getting bankrolled by philanthropic billionaires like Musk, but the route that they took their research: focusing on scaling for all the headline stuff, is insanely expensive, and they needed to make deals with Microsoft to make ends meet. I don't know what the nature of those deals are but I'd imagine it's similar to what deepmind and google get; I find it hard to believe that microsoft will just throw them billions in funding and compute out of the goodness of their hearts.

​

As for Musk, I don't trust the guy; he's very "used car salesman" in how he talks about his work; always overblowing its capability. It erodes credibility and any benefit of doubt I'd give him. Anything Musk says I won't believe until I see it, so I'm not even going to seriously consider him on this topic

damn, thought you were kidding until i googled it :(

I have the opposite view tbh.

Big Tech is under certain constraints; they need to provide value for their shareholders first long before they consider wider public good. Only way I see them funding UBI programs over paying dividends is if the money is pried out of their hands through aggressive tax schemes.

Without this regulation, I see tech companies as pushing us straight towards the (((Bad Future))), where wealth inequality is at an all time high, and all the money in the world aggregates into the hands of a handful of AI oligarchs.

Yes "preprogrammed" definitely is the wrong word here on my part. I'm talking about narrow AI Vs more general AI.

With Optimus, it looks like for each task it has to do must be explicitly preprogrammed. Eg user command: "pick up that ball", it needs to have an explicit navigation task it's trained on, and explicit "grabbing" tasks which then need to be composed by hand and preprogrammed into a routine for retrieving an object. This is as opposed to projects like Google's SayCan where the language of interpreting the task, and the compositionality of prior skills learned to synthesize a policy for solving a problem are all learned.

To me this puts Optimus much closer to Atlas than it does the vision that Musk described last year for robots that can handle highly unstructured environments and custom user tasks

No, based on what Musk was describing, I really thought they'd go the route of LLM conditioned robots like what Google is doing which would be more general, but their tech stack looks to be going very much in the "preprogrammed" direction