Advice Please! New Rescue Dog Potty Training

We are fostering to adopt a 2 year old beagle/Jack Russell mix. He came from a rescue and the rescue got him from North Carolina. He's not crazy about treats and he doesn't care about toys. And I have tried over 6 or 8 different treats and different styles of dog toys. Overall he has a pretty even temper he just wants to cuddle and sniff everything. However i'm finding it frustrating to potty train him. I think he was an outside dog at one time because he is fine going potty outside, however he doesn't understand that he cannot inside. The only place he wont go is in his crate. We already crate him when we are at work and at night and I really would hate to crate him any longer then that. But i cant leave him unsupervised. When he goes outside i tried giving him treats after he goes potty but he is very uninterested and i already give him praise. The times i have caught him going inside i rush him back outside but he just doesn't go anymore. I also have caught him going inside 2 minutes after we just came in.

I hope others have some good advice for you. We adopted adults too, but fortunately they were both potty trained. One thought-from what I've read Beagles can be stubborn with housebreaking in general.

A Beagle cross that doesn't like treats? I didn't think there was such a dog. Do you think that this might mean that he doesn't feel settled yet? Or that you might be giving him too much at mealtimes. Have you tried cheese, or baking some diced liver or chicken pieces.
I do sympathise, as my new Beagle has now been here a month, after spending six years as a kennel dog, and she has also been slower than a puppy to learn that there are house cleanliness rules.
We started by taking her outside at regular intervals, - especially after food and sleep - and staying outside with her until she goes. We then give her a happy 'Yes', and reward her with a treat, but whatever they perceive as being a reward will do (cuddle, tennis ball etc.). I have found that my girl has a roughly four hour interval between wees, so we are overlapping our sleeping times to help her to make the nightime adjustment. It is a fag, but we are 90% there, so will see it through.