I have relatives in Jacksonville, they are still in the cone. So, yeah, you worry.
How do people cope with the threat of a winter ice storm or a storm that might dump a huge amount of snow or plunge into truly arctic temperatures? If you live in the area you learn what to do yet some people die in those events. With a hurricane you get a lot more warning than for an earthquake or tornado. Every local weather broadcaster in the threatened area is telling people how bad it might possibly be. That's not just to be dramatic. Most won't see conditions that bad but some could. So the wise prepare for the worst and hope for the best.
The computer models finally agree on the expected path, though that path is a cone, not a line. The projected steering currents are so weak that it is slowing down (bad because it could dump a lot of rain and storm surge can build) and a slight change could send it off in a different direction, but this close they think they have a good handle on those steering currents. We'll see when this thing actually takes that turn.
Typically the northeast quadrant is the worst for rain and, I think, spawning tornadoes. It depends on the shape of the coast and the direction the storm is moving but the north and northwest quadrant usually have the worst storm surge which can cause a lot of damage. Dorian seems to be following this pattern, Barry earlier this year did not. This does not mean that people on the west or southwest side will get it easy, they won't. But I hope the path keeps the northeast quadrant offshore. So hope for the best but prepare for the worst.