Is this a new furnace? Have you done any work to it recently... new thermostat or anything? Has it ever worked correctly?
When you "turn it off" how are you doing that? Are you using the thermostat and turning the heat setting lower so the thermostat is not calling for the furnace to run?
Make and model will help too.