Via Rail is warning of delays on passenger trains running between Toronto, Ottawa and Montreal on Sunday afternoon.