One of "My Dumb Questions" About "Sin of The Golden Calf"

I'm known for my "dumb questions." I wasn't brought up with the usual Torah stories, and I tend not to take everything as true unless it's straight from the Biblical text.
 

By Nicolas Poussin - http://www.cts.edu/ImageLibrary/Public_domain.cfm, Public Domain, https://commons.wikimedia.org/w/index.php?curid=475193

In the Biblical story of the "Sin of The Golden Calf" in this week's Torah Portion, Ki Tissa, something has been bothering me. I wonder how in a time before reliable watches, clocks, cellphones etc  the recently escaped from slavery Jewish People could have had been so sure of the exact time that they went into a panic when Moses hadn't arrived back at the "right time."

It wasn't all that long ago that it was normal to just wait, because one never knew when exactly someone would arrive, and there was no way to contact them. There could always be justified delays from weather, traffic, unexpected snafus. Way back when, just a few decades ago, we'd take for granted that forty days could be forty-two or even more.

Could the sin be that organizing that calf/idol really started well before the forty day deadline?

What do you think?