German racer Tony Martin, who was on the outside of a tightly packed peloton, was unable to avoid colliding with the sign and the ensuing pileup led to a five-minute delay as riders and bikes were untangled.
Riders briefly halted the race on Tuesday to protest against the danger caused by spectators who were too close to the road.
How did the Tour de France crash even happen? With 100 kilometers remaining, the sextet led the peloton by about two minutes.
Authorities are still looking for the woman, but French publication she got on a flight out of France and is untraceable.
The crash, which occurred on a stretch of the race in which there are no crowd-control barriers, was the first of the day.
The first stage was a 198-kilometer 123-mile trek.