Jon,
I've been down that road once or twice before. You are not alone in your thoughts or situation.
I am in my mid 50s and lost my job of 34 years. After a long search I found a new job that does not allow me much extra time to hunt anymore. The new job does not pay much so I had to take on extra side jobs, plus I have Family commitments that seem to eat up the rest of my time. I spent this past week looking forward to getting out hunting today and came down sick. I have also lost access to much of my previous hunting grounds. I am not looking for sympathy just letting you know that others go through it also. Some folks have plenty of time while others do not. It is just the way it is. One thing that I have attempted to do is take advantage of the long small game season so I at least get out in the woods more. I know it ain't the same as hunting deer but it is something. Best of luck.