Task server not using configuration changes

This is a general question re how/when the task service would see any configuration changes made in the community admin panels

As an example, I disabled the email settings for a site, but the task service kept on trying to send emails

I have also noticed that when a plugin configuration is changed and used by a task job they don't seem to take effect 

Is there a delay before the task service will see changes made in the community configuration or should we restart the task service if any configuration changes are made?