What Sprocket is it?
They usually have a recessed groove where the bolt screws to the crank arm to keep it more flush.
Your top hat needs to be tight too. If it goes in and out by hand then it'll be crap.
The top hats for the Profile sprockets are really tight and need twatting!!
I'd check it by eye when it spins.
Some Cranks cause the sprocket to spin out of alignment when the crank arm bolt is too tight. So tighten up your cranks first and then check the spin and then tighten the crank arm bolt. If it spins true when loose but it pulls out of alignment when tight then maybe a small thin washer between the crank arm and sprocket might allow you to tighten it up fully while still keeping it true. I'd only worry about it once you've fitted everything and checked it with a tensioned chain in place. Some of the mid 90's GT/Profile sprockets are like this!! a lot of OS Sprockets have a slight side to side wobble!!