norcalnatv t1_je3v5yw wrote on March 29, 2023 at 5:42 AM

Reply to Should software developers be concerned of AI? [D] by Chasehud

If they're they're below average they should.

norcalnatv OP t1_j84wt52 wrote on February 11, 2023 at 5:31 PM

Reply to comment by norcalnatv in The Inference Cost Of Search Disruption – Large Language Model Cost Analysis [D] by norcalnatv

If the ChatGPT model were ham-fisted into Google’s existing search
businesses, the impact would be devastating. There would be a $36
Billion reduction in operating income. This is $36 Billion of LLM
inference costs.

norcalnatv OP t1_j84wfs7 wrote on February 11, 2023 at 5:28 PM

Reply to The Inference Cost Of Search Disruption – Large Language Model Cost Analysis [D] by norcalnatv

"Our model is built from the ground up on a per-inference basis, but it lines up with Sam Altman’s tweet and an interview he did recently. We assume that OpenAI used a GPT-3 dense model architecture with a size of175 billion parameters, hidden dimension of 16k, sequence length of 4k,average tokens per response of 2k, 15 responses per user, 13 million daily active users, FLOPS utilization rates 2x higher than FasterTransformer at <2000ms latency, int8 quantization, 50% hardware utilization rates due to purely idle time, and $1 cost per GPU hour. Please challenge our assumptions"

norcalnatv t1_j6drni3 wrote on January 29, 2023 at 5:09 PM

Reply to comment by Biff_Malibu_69 in China’s Top Nuclear-Weapons Lab Used American Computer Chips Decades After Ban by hzj5790

Exactly. Relationships are what it’s all about.

norcalnatv t1_izshj9v wrote on December 11, 2022 at 2:56 PM

Reply to [D] Does Google TPU v4 compete with GPUs in price/performance? by Shardsmp

If you own the laptop it’s always going to be cheaper to use that than going to the cloud.