Big Ben is actually the name of the bell, not the clock tower itself.

You can hear Big Ben chime from across the river on a quiet evening.