Hermit cookie is mostly made with a date and nut filling. Hermit cookies are called so because the date and nut filling hides a little dent in the cookie dough which seems like a little cave. The hermit and his cave come together to give the hermit cookie its name. Some variations of the hermit cookie use raisins instead of dates. Hermit cookies are often flavored with cinnamon, ginger and cloves.