With the elder Gilberts deceased and husband Gobel fighting in World War II, Charlotte became sole caretaker for her six children, ages four to fifteen. Charged with neglect, she failed to show up in court. She was jailed for contempt; her oldest child, Charles, went to jail for “safe keeping.” The Catholic Gilbert children were sent to St. Vincent’s in Vincennes, Indiana, where the five younger siblings remained through the war.