If Brixton is your local club, then that's a great place to start as they're all really friendly and have riders of all levels who can help you get started.
If it's the size of the track that intimidates you, then Peckham is an alternative. Check it out every Saturday morning from 1000 at Bird in Bush Rd, Peckham.
BADBMX (Dagenham) is also a great club to check out.