Distance from the epicenter matters as well. I was 6 miles from a 4.9 last month, my first real earthquake. It was unsettling and concerning (including for the people I was on zoom with) but that was it. Nothing fell off the walls, iirc no fatalities or buildings demolished.

The scale is logarithmic. My rule of thumb is that 5 is where things start to get much more dangerous.