Bruce's heart leaps into his throat immediately. Clint doesn't look concerned, but that doesn't really mean anything at all; for all Bruce knows he could be hiding some intense and life-threatening knife wound to the back when he casually strolls in and asks what's for dinner. Clint's calm does nothing to quell Bruce's alarm at the extremely ambiguous statement, but he stands up without a rush, and he forces his hands into his pockets, instead of clutching them in front of him.

"Oh yeah? Was it a friendly ghost, right hooks aside? Or should we look up the Ghostbusters?"

The idea that Clint ran into someone he knew is not good, very not good. They might have to pack up and leave now, which is always a vulnerable situation, and where would they go? It's easier to run solo; running with Clint, even though Clint's no stranger to this, is harder because it matters more, now. He's always afraid of being captured, but seeing Clint captured would be far worse.