I always eat the heart. The flavor of the heart may be the best on the deer. What is unusual about heart is the meat doesn't seem to have a grain, thus the chewyness if overcooked.
I don't eat the liver anymore. Liver has it's pro's and con's, with too many con's. Taste, being a big con. Being merely tolerable in taste isn't enough reason for me to eat an organ or anything else for that matter.
Liver is high in protein and vitamins, but also very high in cholesterol and iron. Heart patients are advised to stay away from liver, even from supposedly "lean" animals, like deer and elk.
If I want to blow my cholesterol watch on a splurge meal, I'll do it with a big pizza or bacon cheeseburger instead of a liver, with even much higher cholesterol than most "fatty foods". I can eat 5-6 BK Whoppers for the same cholesterol content as a liver.
Plus the liver is the filter of everything. If there is any concern of any old industrial waste residue, like we have in some fish, then it will be in the liver of land animals too.
In parts of Michigan, due to our industrial past, our DNR even has warnings NOT to eat the liver, for the above reason. My hunch is, there's lots of little area's around where nasty industrial stuff was dumped in the ground or water 50 years ago that ends up the liver of critters we eat.
FROM MICHIGAN DNR...
http://www.michigan.gov/dnr/0,1607,7-153-10363_10856_10905-171817--,00.html Dioxin Advisory Information
Health assessors from the Michigan Department of Community Health (MDCH) and Michigan Department of Natural Resources and Environment (DNRE) determined that samples of wild game taken in 2003, 2004 and 2007 from the floodplains of the Tittabawassee River and Saginaw River downstream of Midland contain high levels of dioxin and dioxin-like compounds. Wild game that have been tested include deer, turkey, cottontail rabbit, squirrel, wood duck and Canada goose. As a result, the MDCH advises that hunters and their families follow these recommendations related to deer:
Do not eat the liver from deer harvested in or near the Tittabawassee River floodplain downstream of Midland. Eating liver taken from deer harvested in the floodplain of the Saginaw River is not likely to result in adverse health effects.
Limit consumption of muscle meat from deer harvested in or near the floodplain of the Tittabawassee River downstream of Midland and in or near the floodplain of the Saginaw River. Women of childbearing age and children under the age of 15 should eat only one meal of deer muscle meat harvested in the floodplains per week. Trimming any visible fat will lower the level of dioxins in the cooked meat.
Other wild game that have not been tested in this area may also contain dioxins at levels that are a concern. To reduce general dioxin exposure from other wild game, trim any visible fat from the meat before cooking, do not consume organ meats such as the liver or brains, and do not eat the skin.