There was a famous incident in the early 1970's where an actual EBS alert got sent out by mistake instead of the weekly test. (there is audio out there online of an announcer at WOWO in Fort Wayne vamping nervously for several minutes until the All Clear was sent out over the wires.)
It was determined that the vast majority of stations did NOT break into programming, as the employees on duty looked at that and were incredulous. They figured it HAD to be a mistake. This sort of defeats the whole purpose of having an EBS, which ultimately led to today's EAS that removes the human element entirely.