Housing Assistance Conflict

My future wife and I are buying a house. We were approved by a credit union (not a fan) and started the process of finding where and what we wanted to live in. We found an amazing house and both agreed to put an offer down. We got accepted! Over the next week we heard from other people of assistance programs for down payment and closing costs. Instantly I tried to separate myself from them because of beliefs, but I am now having internal conflict with myself because I know I will never be able to forgive myself for using the money that was stolen from other people to help myself. The terms do require that we pay them back at the end of our time at the house but I do not think I will feel like this is 100% our house because we are using some assistance. It is also going to be hard to explain to people the principals of Liberty to people because I think we are going to use this program. I don't feel like a good Libertarian and at this point I think it is a disservice to identify as one. Fuck.