Oh, for sure. Pithy phrases only get you so far. But in a broad, hand-wavy sense, I use this analysis:

"Does this person enrich my life, and do I enrich theirs?"

Enrichment could be alternative view points, or activities together, or being able to help someone who values the help. But even that basic test, earlier in my life, brought me to the startling realization that my social circle just WASN'T. I wasn't growing, and I was frequently uncomfortable with the status of my relationships, romantic or friendly. Finding people who made me better, and who (hopefully) I make better drastically changed my quality of life for the better. Most of my family didn't make the cut, and my life is SUBSTANTIALLY better without them. The family that is still around is valued that much more.