Why do people believe atheists are devil worshipers?

I must not be the only who knows people who equate atheism with devil worship. Does anyone have an idea where such a notion come from? Am I misunderstanding atheism? Doesn't it mean the lack of belief in God, god, gods, deities, angels, fallen angels, cast out angels, the devil, and such? How could people worship an entity they don't believe in?

Edited for spelling typo from believe to belief.