It's possible, especially if companies start marketing it as a replacement to search engines. We've all seen how these systems can get things factually wrong. Hell, Google's first demo contained a factual error. So if they are presented as a place to get factual information, and people start asking medical questions that they get wrong answers to, that could cause real harm.
OllaniusPius t1_j7rxrhs wrote
Reply to comment by malcolmrey in I asked Microsoft's 'new Bing' to write me a cover letter for a job. It refused, saying this would be 'unethical' and 'unfair to other applicants.' by TopHatSasquatch
It's possible, especially if companies start marketing it as a replacement to search engines. We've all seen how these systems can get things factually wrong. Hell, Google's first demo contained a factual error. So if they are presented as a place to get factual information, and people start asking medical questions that they get wrong answers to, that could cause real harm.