Depleted Robins, without Hans Andersen and Jason Doyle, won for the second time this season at Perry Barr last Wednesday in a thrilling last heat decider.
But Rossiter is warning his riders they will be up against a Birmingham side desperate for revenge.
He said: “We can’t take anything for granted because they will be wounded and looking to get back at us.
After a monumental battle for the top spot, Poole eventually got the better of Swindon – but Rossiter feels that may work in favour of his side.
He said: “It would have been nice to finish top of the league, but it maybe takes a bit of pressure off us because Poole are certainly going in as the favourites.”
“We need to be wary of Birmingham bouncing back but we’re confident this team will beat anyone on top form.”
The second leg is at the Abbey Stadium next Monday, October 1.
BIRMINGHAM: Bjarne Pedersen, Seb Ulamek, Danny King, Ben Barker, Josh Auty, Martin Smolinski, Nicolas Covatti.
SWINDON: Hans Andersen, Nick Morris, Peter Kildemand, Jason Doyle, Troy Batchelor, Simon Stead, Dawid Lampart.