Viewing a single comment thread. View all comments

9182763498761234 t1_je8wfs4 wrote

Reply to comment by dreaming_geometry in [R] LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention by floppy_llama

Work in a niche field instead! There are hundreds of smaller topics in ML that are yet unexplored and only a couple people working on it. I’m working on one of those and it is awesome. The field is slowly progressing but slowly enough that I can make a valuable contribution without getting scooped all the time.

24

silva_p t1_je9aw63 wrote

Can you share your niche?

4

saintshing t1_je9e5q1 wrote

Natural language processing for cats and dogs

18

currentscurrents t1_jean0il wrote

Other researchers are working on an LLM for whales.

Looks feasible to me, whale calls are no more alien to the computer than English is. The hard part is collecting enough data.

5

DigThatData t1_jeb49b8 wrote

this is probably not a concern for whale vocalizations, but an issue for attempting to decode animal communications generally via LLMs is that they're probably communicating as much information (if not more) non-vocally. for example, if we wanted to train an LLM to "understand" dog communication, it'd probably be more important to provide it with signals corresponding to changes in body and face pose than vocalizations. interesting stuff in any event.

3

currentscurrents t1_jeb4shv wrote

Yeah, I think that's why they're starting with whales - they're an easy subject since their vocalizations can be heard through the water from miles away. They also seem to have a fairly complex vocal language, unlike for example songbirds with memorized mating calls.

2

Koda_20 t1_jecddbs wrote

I feel like they are starting with whales because it generates more publicity because Nemo lol

They are probably not but I thought it was funny

2