Hello Joe,
I believe you are talking about this story: Permission to Travel Letters and Why You Need Them.
Your fears may well be unfounded, I don’t know your personal situation. However, according to the law in the U.S., neither you nor your wife nor any other adult could take a child under 18 outside the U.S. by plane, land or ship (there are some exceptions for recreational cruises) without the consent of that child’s birth parents or legal guardians. In this country, that law is supposed to be enforced by the carrier — therefore the plane, ground transportation or ship company are supposed to collect a notarized Permission to Travel Letter from the adult(s) accompanying the child or else, not allow those passengers to board.
Please let us know if you have any other questions and safe travels!