Diet question regarding LDL and HDL

Hello everyone first time poster. A few months ago I got bloodwork done and was informed my cholesterol was high for my age (31 male). So I immediately started to change my eating habits. I cut out all fast food, lived off chicken and lean meats, and have been working out every other day for 3 months. My first test my LDL was 182 and my HDL was 44(Total Cholesterol came back as 255). Got my results back today and now my LDL is 187 and my HDL is 39(Total cholesterol is now 250).

Should I be alarmed? Im not sure exactly what ive been doing wrong. Im someone with a lot of anxiety (on medicine for it) and this is making me fearful of eating anything remotely bad.

Should I be as stressed as I am about this? I feel my doctor is going to put me on medicine but I want to avoid it if possible.