professorlust t1_jccjn4t wrote
Reply to comment by Caskla in [D] What do people think about OpenAI not releasing its research but benefiting from others’ research? Should google meta enforce its patents against them? by [deleted]
If you can’t replicate their results, then they’re not useful for research
VelveteenAmbush t1_jcd760v wrote
They're purposefully withholding the information you'd need to use their results in research. This proposed research boycott is sort of a "you can't fire me, I quit" response.
professorlust t1_jcddvx5 wrote
Agreed
eposnix t1_jccpohv wrote
How many companies could realistically replicate their results though? We already have a pretty good idea of what's going on under the hood, but would knowing the intricacies of GPT-4 help anyone smaller than Google?
professorlust t1_jcdeiux wrote
The argument from a research perspective is that scale isn’t likely the Holy Grail.
It’s undoubtedly important, yes.
BUT for a researcher, the quest is to determine how important scale truly is AND how to determine ways that help reduce dependence on scale.
BrotherAmazing t1_jcdloe7 wrote
You can still replicate results in private under a non-disclosure agreement or verify/validate results without it getting published to the world though.
I like open research but research that happens in private still can be useful and is reality.
professorlust t1_jce19sb wrote
What researcher is signing an NDA?
That’s literally the opposite of what replication research is supposed to accomplish.
Operating under an NDA is for primary research, not replication
BrotherAmazing t1_jce3zky wrote
I would be happy to sign an NDA if Google allowed me to have access to verify, validate, and run some of their most prized models they keep secret and have not released, and it is incredibly rare for an NDA to last forever.
Also, a lot of research goes on behind closed doors among people who have signed NDAs. They still replicate each other’s work and verify and validate it, they just don’t publish it for you to read.
This thread isn’t specifically about “replication research” across the broad range international community either, is it? OP did not indicate that, and primary research a company performs and then successfully transitions it into a system that empirically outperforms the competition is validation enough that need not be replicated by their competitors. In fact, the whole point is you don’t want anyone to replicate it but it is still did valid useful research if you bring a product to market that everyone demands and finds useful.
When you work for Google or nearly any company and nove away from academia, you don’t have an ability to publish everything the company ever has done that you learn about or everything you do at the company automatically. Are you really under that impression? Have you ever worked in the Corporate world??
professorlust t1_jce4rv6 wrote
Check out Axriv if you think there’s only academic researchers publishing
BrotherAmazing t1_jch1gll wrote
I never said they don’t publish, re-read.
I can tell you firsthand what they publish has to get approval, and a lot of things do jot get approval to publish and are held as trade secrets. It boggles my mind this sub clearly has so many people who have never worked on the Corporate side of this industry and have these strong ideas that the Corporate side is or has ever been fully transparent and allows employees to publish anything and everything. The is so far from the truth it’s not funny.
For every model and paper published, there exists another model and many other papers that are not approved to be published and many exist in a different format as internal publications only. Other internal publications get watered down and a lot of extra work is omitted in order to get approval to publish. or they publish “generation 3” to the world while they’re working on “generation 5” internally.
Viewing a single comment thread. View all comments