TThhee BBeerrkkeelleeyy UUNNIIXX((RR)) TTiimmee SSyynncchhrroonniizzaattiioonn PPrroottooccooll _R_i_c_c_a_r_d_o _G_u_s_e_l_l_a_, _S_t_e_f_a_n_o _Z_a_t_t_i_, _a_n_d _J_a_m_e_s _M_. _B_l_o_o_m Computer Systems Research Group Computer Science Division Department of Electrical Engineering and Computer Science University of California, Berkeley Berkeley, CA 94720 IInnttrroodduuccttiioonn The Time Synchronization Protocol (TSP) has been designed for specific use by the program _t_i_m_e_d, a local area network clock synchronizer for the UNIX 4.3BSD operating system. Timed is built on the DARPA UDP protocol [4] and is based on a master slave scheme. TSP serves a dual purpose. First, it supports messages for the synchronization of the clocks of the various hosts in a local area network. Second, it supports messages for the election that occurs among slave time daemons when, for any reason, the master disappears. The synchronization mechanism and the election procedure employed by the program timed are described in other documents [1,2,3]. Briefly, the synchronization software, which works in a local area network, consists of a collection of _t_i_m_e _d_a_e_m_o_n_s (one per machine) and is based on a master-slave structure. The present implementation keeps processor clocks synchro- nized within 20 milliseconds. A _m_a_s_t_e_r _t_i_m_e _d_a_e_m_o_n measures the time difference between the clock of the machine on which it is running and those of all other machines. The current implementation uses ICMP _T_i_m_e _S_t_a_m_p _R_e_q_u_e_s_t_s [5] to measure the clock difference between machines. The master computes the _n_e_t_w_o_r_k _t_i_m_e as the average of the times pro- vided by nonfaulty clocks.1 It then sends to each _s_l_a_v_e _t_i_m_e ----------- This work was sponsored by the Defense Advanced Research Projects Agency (DoD), monitored by the Naval Electronics Systems Command under contract No. N00039-84-C-0089, and by the Italian CSELT Corporation. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the Defense Research Projects Agency, of the US Gov- ernment, or of CSELT. 1 A clock is considered to be faulty when its SMM:12-2 The Berkeley UNIX Time Synchronization Protocol _d_a_e_m_o_n the correction that should be performed on the clock of its machine. This process is repeated periodically. Since the correction is expressed as a time difference rather than an absolute time, transmission delays do not interfere with synchronization. When a machine comes up and joins the network, it starts a slave time daemon, which will ask the master for the correct time and will reset the machine's clock before any user activity can begin. The time daemons therefore maintain a single network time in spite of the drift of clocks away from each other. Additionally, a time daemon on gateway machines may run as a _s_u_b_m_a_s_t_e_r. A submaster time daemon functions as a slave on one network that already has a master and as master on other networks. In addition, a submaster is responsible for propagating broadcast packets from one network to the other. To ensure that service provided is continuous and reli- able, it is necessary to implement an election algorithm that will elect a new master should the machine running the current master crash, the master terminate (for example, because of a run-time error), or the network be partitioned. Under our algorithm, slaves are able to realize when the master has stopped functioning and to elect a new master from among themselves. It is important to note that since the failure of the master results only in a gradual diver- gence of clock values, the election need not occur immedi- ately. All the communication occurring among time daemons uses the TSP protocol. While some messages need not be sent in a reliable way, most communication in TSP requires reliability not provided by the underlying protocol. Reliability is achieved by the use of acknowledgements, sequence numbers, and retransmission when message losses occur. When a mes- sage that requires acknowledgment is not acknowledged after multiple attempts, the time daemon that has sent the message will assume that the addressee is down. This document will not describe the details of how reliability is implemented, but will only point out when a message type requires a reli- able transport mechanism. The message format in TSP is the same for all message types; however, in some instances, one or more fields are not used. The next section describes the message format. The following sections describe in detail the different mes- sage types, their use and the contents of each field. NOTE: The message format is likely to change in future versions of ----------- value is more than a small specified interval apart from the majority of the clocks of the machines on the same network. See [1,2] for more details. The Berkeley UNIX Time Synchronization Protocol SMM:12-3 timed. MMeessssaaggee FFoorrmmaatt All fields are based upon 8-bit bytes. Fields should be sent in network byte order if they are more than one byte long. The structure of a TSP message is the following: 1) A one byte message type. 2) A one byte version number, specifying the protocol ver- sion which the message uses. 3) A two byte sequence number to be used for recognizing duplicate messages that occur when messages are retransmitted. 4) Eight bytes of packet specific data. This field con- tains two 4 byte time values, a one byte hop count, or may be unused depending on the type of the packet. 5) A zero-terminated string of up to 256 ASCII characters with the name of the machine sending the message. The following charts describe the message types, show their fields, and explain their usages. For the purpose of the following discussion, a time daemon can be considered to be in one of three states: slave, master, or candidate for election to master. Also, the term _b_r_o_a_d_c_a_s_t refers to the sending of a message to all active time daemons. AAddjjttiimmee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | SSeeccoonnddss ooff AAddjjuussttmmeenntt | +------------------------------------------------------+ | MMiiccrroosseeccoonnddss ooff AAddjjuussttmmeenntt | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_ADJTIME (1) SMM:12-4 The Berkeley UNIX Time Synchronization Protocol The master sends this message to a slave to communicate the difference between the clock of the slave and the net- work time the master has just computed. The slave will accordingly adjust the time of its machine. This message requires an acknowledgment. AAcckknnoowwlleeddggmmeenntt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_ACK (2) Both the master and the slaves use this message for acknowledgment only. It is used in several different con- texts, for example in reply to an Adjtime message. MMaasstteerr RReeqquueesstt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_MASTERREQ (3) The Berkeley UNIX Time Synchronization Protocol SMM:12-5 A newly-started time daemon broadcasts this message to locate a master. No other action is implied by this packet. It requires a Master Acknowledgment. MMaasstteerr AAcckknnoowwlleeddggeemmeenntt | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_MASTERACK (4) The master sends this message to acknowledge the Master Request message and the Conflict Resolution Message. SSeett NNeettwwoorrkk TTiimmee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | SSeeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMiiccrroosseeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_SETTIME (5) The master sends this message to slave time daemons to set their time. This packet is sent to newly started time daemons and when the network date is changed. It contains SMM:12-6 The Berkeley UNIX Time Synchronization Protocol the master's time as an approximation of the network time. It requires an acknowledgment. The next synchronization round will eliminate the small time difference caused by the random delay in the communication channel. MMaasstteerr AAccttiivvee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_MASTERUP (6) The master broadcasts this message to solicit the names of the active slaves. Slaves will reply with a Slave Active message. SSllaavvee AAccttiivvee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_SLAVEUP (7) The Berkeley UNIX Time Synchronization Protocol SMM:12-7 A slave sends this message to the master in answer to a Master Active message. This message is also sent when a new slave starts up to inform the master that it wants to be synchronized. MMaasstteerr CCaannddiiddaattuurree MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_ELECTION (8) A slave eligible to become a master broadcasts this message when its election timer expires. The message declares that the slave wishes to become the new master. CCaannddiiddaattuurree AAcccceeppttaannccee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_ACCEPT (9) SMM:12-8 The Berkeley UNIX Time Synchronization Protocol A slave sends this message to accept the candidature of the time daemon that has broadcast an Election message. The candidate will add the slave's name to the list of machines that it will control should it become the master. CCaannddiiddaattuurree RReejjeeccttiioonn MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_REFUSE (10) After a slave accepts the candidature of a time daemon, it will reply to any election messages from other slaves with this message. This rejects any candidature other than the first received. MMuullttiippllee MMaasstteerr NNoottiiffiiccaattiioonn MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_CONFLICT (11) The Berkeley UNIX Time Synchronization Protocol SMM:12-9 When two or more masters reply to a Master Request mes- sage, the slave uses this message to inform one of them that more than one master exists. CCoonnfflliicctt RReessoolluuttiioonn MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_RESOLVE (12) A master which has been informed of the existence of other masters broadcasts this message to determine who the other masters are. QQuuiitt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_QUIT (13) This message is sent by the master in three different contexts: 1) to a candidate that broadcasts an Master SMM:12-10 The Berkeley UNIX Time Synchronization Protocol Candidature message, 2) to another master when notified of its existence, 3) to another master if a loop is detected. In all cases, the recipient time daemon will become a slave. This message requires an acknowledgement. SSeett DDaattee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | SSeeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMiiccrroosseeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_SETDATE (22) The program _d_a_t_e(1) sends this message to the local time daemon when a super-user wants to set the network date. If the local time daemon is the master, it will set the date; if it is a slave, it will communicate the desired date to the master. SSeett DDaattee RReeqquueesstt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | SSeeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMiiccrroosseeccoonnddss ooff TTiimmee ttoo SSeett | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_SETDATEREQ (23) The Berkeley UNIX Time Synchronization Protocol SMM:12-11 A slave that has received a Set Date message will com- municate the desired date to the master using this message. SSeett DDaattee AAcckknnoowwlleeddggmmeenntt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_DATEACK (16) The master sends this message to a slave in acknowledg- ment of a Set Date Request Message. The same message is sent by the local time daemon to the program _d_a_t_e_(_1_) to con- firm that the network date has been set by the master. SSttaarrtt TTrraacciinngg MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_TRACEON (17) The controlling program _t_i_m_e_d_c sends this message to the local time daemon to start the recording in a system SMM:12-12 The Berkeley UNIX Time Synchronization Protocol file of all messages received. SSttoopp TTrraacciinngg MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_TRACEOFF (18) _T_i_m_e_d_c sends this message to the local time daemon to stop the recording of messages received. MMaasstteerr SSiittee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_MSITE (19) _T_i_m_e_d_c sends this message to the local time daemon to find out where the master is running. The Berkeley UNIX Time Synchronization Protocol SMM:12-13 RReemmoottee MMaasstteerr SSiittee MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_MSITEREQ (20) A local time daemon broadcasts this message to find the location of the master. It then uses the Acknowledgement message to communicate this location to _t_i_m_e_d_c. TTeesstt MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_TEST (21) For testing purposes, _t_i_m_e_d_c sends this message to a slave to cause its election timer to expire. NOTE: _t_i_m_e_d is not normally compiled to support this. SMM:12-14 The Berkeley UNIX Time Synchronization Protocol LLoooopp DDeetteeccttiioonn MMeessssaaggee | | | BByyttee 11 | BByyttee 22 | BByyttee 33 | BByyttee 44 +------------+-------------+-------------+-------------+ | TTyyppee | VVeerrssiioonn NNoo.. | SSeeqquueennccee NNoo.. | +------------+-------------+---------------------------+ | HHoopp CCoouunntt | (( uunnuusseedd )) | +------------+-----------------------------------------+ | (( uunnuusseedd )) | +------------------------------------------------------+ | MMaacchhiinnee NNaammee | +------------------------------------------------------+ | .. .. .. | +------------------------------------------------------+ Type: TSP_LOOP (24) This packet is initiated by all masters occasionally to attempt to detect loops. All submasters forward this packet onto the networks over which they are master. If a master receives a packet it sent out initially, it knows that a loop exists and tries to correct the problem. RReeffeerreenncceess 1. R. Gusella and S. Zatti, _T_E_M_P_O_: _A _N_e_t_w_o_r_k _T_i_m_e _C_o_n_- _t_r_o_l_l_e_r _f_o_r _D_i_s_t_r_i_b_u_t_e_d _B_e_r_k_e_l_e_y _U_N_I_X _S_y_s_t_e_m, USENIX Summer Conference Proceedings, Salt Lake City, June 1984. 2. R. Gusella and S. Zatti, _C_l_o_c_k _S_y_n_c_h_r_o_n_i_z_a_t_i_o_n _i_n _a _L_o_c_a_l _A_r_e_a _N_e_t_w_o_r_k, University of California, Berkeley, Technical Report, _t_o _a_p_p_e_a_r. 3. R. Gusella and S. Zatti, _A_n _E_l_e_c_t_i_o_n _A_l_g_o_r_i_t_h_m _f_o_r _a _D_i_s_t_r_i_b_u_t_e_d _C_l_o_c_k _S_y_n_c_h_r_o_n_i_z_a_t_i_o_n _P_r_o_g_r_a_m, University of California, Berkeley, CS Technical Report #275, Dec. 1985. 4. Postel, J., _U_s_e_r _D_a_t_a_g_r_a_m _P_r_o_t_o_c_o_l, RFC 768. Network Information Center, SRI International, Menlo Park, Cal- ifornia, August 1980. 5. Postel, J., _I_n_t_e_r_n_e_t _C_o_n_t_r_o_l _M_e_s_s_a_g_e _P_r_o_t_o_c_o_l, RFC 792. Network Information Center, SRI International, Menlo Park, California, September 1981.