Playing sound causes drop on FPS

So, when I use; the fps on my game goes from 60 to about 40 and stays that way for a few seconds, then goes back to 60. Doesnt metter how many sound effects are beeing played at a time, fps does not go below 40~35, but even if theres only one sound beeing played it drops the frame rate.
Playing sounds should not cause drops on frame rate, right?
What am I doing wrong?