ChuckSeven t1_jedsgz5 wrote on March 31, 2023 at 8:07 AM

Reply to comment by MysteryInc152 in [D] Can large language models be applied to language translation? by matthkamis

I know about this post. It is interesting but the results here are far from conclusive. The BLOOM papers also did translation experiments and they say "... In the one-shot setting, BLOOM can, with the right prompt, perform competent translation, although it is behind dedicated (supervised) models such as M2M-100".

So let's maybe use some quantifiable measures instead of just looking at a few cherry-picked examples and claim otherwise?

MysteryInc152 t1_jee5zba wrote on March 31, 2023 at 11:12 AM

It's not cherry picked lol.

Wild how everyone will just use that word even when they've clearly not tested the supposed model themselves. I'm just showing you what anyone who's actually used these models for translation will tell you

https://youtu.be/5KKDCp3OaMo

https://www.reddit.com/r/visualnovels/comments/11rty62/gpt4_ai_vs_human_translation_on_the_opening_scene/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button

ChuckSeven t1_jeeae4o wrote on March 31, 2023 at 11:57 AM

Look, it doesn't matter. You can't claim that LLM are better if you don't demonstrate it on an established benchmark with a large variety of translations. How should I know if those Japanese anime translations are correct? For what its worth it might be just "prettier" text but a wrong translation.

It's sad to get downvoted on this subreddit for insisting on very basic academic principles.

MysteryInc152 t1_jeecbeq wrote on March 31, 2023 at 12:15 PM

I didn't downvote you but it's probably because you're being obtuse. anyway whatever. if you don't want to take evidence at plain sight then don't. the baseline human comparisons are right there. Frankly it's not my problem If you're so suspicious of results and not bilingual to test it yourself. It's not really my business if you believe me or not.

ChuckSeven t1_jeenkvs wrote on March 31, 2023 at 1:46 PM

I'm happy to take evidence into account. Your results indicate that LLM can be beneficial for translation. As I said previously, it looks interesting. But you claim, and I quote: "They re far superior to current sota" solely based on your personal and human comparison. This is an over-generalisation and not scientific. Like a flat earther claiming the earth is flat because .. just look at it "evidence at plain sight".