Good idea.
Keep him away. Let the kids talk to him on the phone about when they see him. And don't even let him pull up in the driveway.

He blew his chance to be the father who hangs out with them and you in the same house.

Now he can hang out with them in his own place.

His demands are amazing.
He would push no matter what you did. Forget about making him happy/satisfied with the agreement.

Do what you have to for you and your family.
Mari