Cookie Authentication should be used when you have an existing login system which you want to use for the community, and you want your users to be able to sign in once and be logged in to all your websites, rather than having to re-authenticate with each website individually. For example, your main website may already have a login system, and you want users to be logged in to the community automatically when they log in to your main website (and vice versa).
When Cookie SSO is enabled, the login & registration URLs for the community are redirected to point to the external authentication system and when the external login system logs a user in, a cookie is set that allows the community to log the user in, and create their account if it doesn't exist in the community. This change generally requires developer involvement to create the cookie that Telligent Community can read to implement the single sign-on behavior.
For more details about cookie-based single sign-on, see the developer topic: Cookies Authentication.