Asymptomatic infections can create damage and it's a very real problem.

For example, it's a concern that there are observed, enduring differences/abnormalities in the lungs of even asymptomatic COVID-19 convalescents that could greatly complicate future treatments, e.g. here in radiation therapy for certain cancers:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7462877/

Along with more publicized "long COVID" sequelae, these kinds of things are why the myopic focus on fatality rates throughout the pandemic has been so stupid; for every already-at-risk person killed by a primary infection, there are hundreds, thousands, or even more who are going to experience some degree of complexity in the onset, management, and treatment of future problems, many of those patients having noted little or no acute symptomology during primary infection.