>"Work in the office" is just the manager's way of saying they don't understand what you do and have no way to measure your output unless they can constantly look over at your desk and make sure you're not happy.

More like, it's the manager's way of saying "my manager is starting to notice my job serves no purpose when I can't roam the halls like a freak, badgering employees and picking up buzzwords to drop into conversations, and his manager is starting to notice, too."