Madeleine: why doesn't David Payne sue?

If what is been said is true about him, just why would you take him on a family holiday with you when your children are so young. I dont know this case just keeps getting stranger by the day :)