Facebook has launched its own time keeping service to help keep devices in sync over the internet – a problem that fascinates both engineers who have to manage infrastructure fleets and big name tech vendors keen to put their own time stamp on the world.

As the social networking giant explains in a rather deep blog, the clocks in many of the devices connected to the net “contain inaccurate internal oscillators, which can cause seconds of inaccuracy per day and need to be periodically corrected.”

So, “devices all over the world rely on Network Time Protocol (NTP) to stay synchronized to a more accurate clock over packet-switched, variable-latency data networks”.

Unsurprisingly, Facebook’s engineers “need to know the accurate time difference between two random servers in a data center so that datastore writes don’t mix up the order of transactions. We need to sync all the servers across many data centers with sub-millisecond precision.” Things are further complicated by the concept of smeared leap seconds – the “means to change the time in very small increments across multiple hours” needed to take account for the Earth’s refusal to rotate precisely.

So, Facebook took a deep look at “chrony, a modern NTP server implementation with interesting features.” The blog details how the company tested its approach, weighing up chrony against ntpd, and the steps it took to narrow the “offsets” – or estimated differences between devices – to bring “internal precision to the sub-millisecond level”.

It found that chrony is “significantly more accurate and scalable than the previously used service, ntpd, which made it an easy decision for us to replace ntpd in our infrastructure”.

And, therefore it is also the “foundation of our Facebook public NTP service, available from time.facebook.com”.

The social giant said its NTP service is built in four layers. First is “a layer of satellites with extremely precise atomic clocks from a global navigation satellite system (GNSS), such as GPS, GLONASS, or Galileo.” Layer two is “Facebook atomic clock synchronizing with a GNSS”.

Next comes a pool of NTP servers syncing to layer two, which features leap second smearing. The fourth layer is “a tier of servers configured for a larger scale. They receive smeared time and are ignorant of leap seconds.”

Summarising what it learned, Facebook said “measuring time is challenging” and that “both ntpd and chrony provide estimates that are somewhat true.”

“In general, if you want to monitor the real offset, we recommend using 1PPS or external devices with GNSS receivers and an atomic clock,” 

But, it concluded “In comparing ntpd with chrony, our measurements indicate that chrony is far more precise, which is why we’ve migrated our infra to chrony and launched a public NTP service. We’ve found that the effort to migrate is worth the immediate improvement in precision from tens of milliseconds to hundreds of microseconds.”

Google launched its own NTP service in 2016, and Apple, Microsoft and HPE all offer their own take on time.

Unfortunately, given the shocks roiling the world right now, like Cher, none of them have cracked the problem of how to turn back time.