Documentation forKiwi Syslog Server NG

The Kiwi Reliable Delivery Protocol (KRDP)

This documentation is for legacy Kiwi Syslog Server versions 9.8.2 and older.

The Kiwi Reliable Delivery Protocol was designed to solve the problem of losing data when a TCP connection is broken due to a network failure.

KRDP uses the TCP protocol as the underlying transport. This ensures that each packet sent is sequenced and acknowledged when received. The TCP protocol on the receiving system handles the packet order and ensures that any missing packets are resent.

The problem

TCP works well as a reliable transport when the connection can be opened and closed cleanly. During a TCP close handshake, any outstanding packets are usually received and acknowledged before the connection is closed.

However, if a break in the network occurs during message sending, the sender will continue to send packets until the TCP window size is reached. When no acknowledgment is received after a timeout period, the Winsock stack will fire a timeout event. When this happens, it is not possible to know exactly which message (or part message) was last received and acknowledged by the remote end. Any data that was sitting in the Winsock stack's buffer will be lost. Depending on the TCP window size and the speed of the data being sent, this could be hundreds of lost messages.

The solution

KRDP works by adding another acknowledgment and sequencing layer over the top of the TCP transport. KRDP wraps each syslog message with a header which contains a unique sequence number. The KRDP sender keeps a local copy of each message it has sent. The KRDP receiver periodically acknowledges receipt of the last KRDP wrapped syslog message it has received. The KRDP sender can then remove all locally stored messages up to the last acknowledged sequence number. When the connection is broken and re-established, the receiver informs the sender which messages need to be resent.

Each KRDP sender is identified with a unique connection name. This allows the sender and receiver to reestablish the same session and sequence numbers, even if the IP address or sending port of the sender has been changed due to DHCP etc.

Unique message sequencing

Each KRDP message is identified with a unique sequence number. The sequence starts at 1 and increments in steps of 1 up to 2147483647 (2 billion), then wraps around to 1 again. The message number 0 is used to indicate that the system does not know the last sequence number and that it has had to assume a fresh start. If this occurs, both the sender and receiver will log an error to note the lost messages.

Dealing with international characters

Unicode allows the mapping of all international character sets into a known byte sequence. The mapping of non US-ASCII characters requires the use of more than a single byte per character. The most commonly used way of sending these multi-byte characters over TCP is to use UTF-8 encoding. The KRDP sender will encode the syslog messages as UTF-8 and the KRDP receiver will decode them back to Unicode again.

The KRDP message format

  • Sender (S)
  • Receiver (R)

Message Types (MsgType)

00 = SenderID

01 = ReceiverResponse

02 = Sequenced message

03 = Message acknowledgement

04 = Receiver KeepAlive

99 = Error message

Message format

KRDP AA 0000000000 Message<CR>

KRDP = Unique tag

Space (ASCII 32)

AA = Msg type (as above)

Space (ASCII 32)

0000000000 = Sequence number 0 to 2147483647

Space (ASCII 32)

Message = UTF-8 encoded message text

<CR> = Carriage return character ASCII 13 to indicate end of message stream

Sequence of events

S connects via TCP

S sends first ID packet (MsgType 00)

R responds with ReceiverResponse message (MsgType 01)

S sends sequenced messages (MsgType 02)

Rules

  1. If the first message R receives is not a ID message (MsgType 00), R disconnects. (Any data received is ignored).
  2. If R does not receive ID message after 60 seconds, R disconnects.
  3. After S sends the ID message, S will wait up to 60 seconds for a ReceiverResponse message. If there is no response, S will disconnect session.
  4. R sends ACK messages to S with the next expected message sequence.
  5. ACK messages are sent no more frequently than once every 200ms.

Message formats

MsgType 00 (Version and SenderID)

KRDP 00 PV UniqueKey<CR>

The unique key identifies the channel and is used to synchronise the message numbers

PV = Protocol Version to use. 01 = KRDP Reliable/Acknowledged

Unique key format is free form.

An example would be: "IP=192.168.1.1, Host=myhost.com, ID=Instance1"

Or, just: "Instance1"

Since the receiver might already have an "Instance1" name from another source, the first UniqueKey would

be better. Use as much information to uniquely describe the source of the messages

MsgType 01 (ReceiverResponse message)

KRDP 01 0000000000 Listener ID<CR>

Message number is 10 digit number 0000000000 to 2147483647

MsgType 02 (Sender Message content)

KRDP 02 0000000000 Message content<CR>

Message number is 10 digit number 0000000000 to 2147483647

MsgType 03 (Receiver ACK)

KRDP 03 0000000000 ACK<CR>

Message number is 10 digit number 0000000000 to 2147483647

Message number indicates the next sequence number it expects to receive

ACK messages are sent at a maximum rate of once every 200ms

MsgType 04 (Keep alive)

KRDP 04 0000000000 KeepAlive<CR>

Message number is 10 digit number 0000000000 to 2147483647

Message number = Next expected message number

If being sent by Sender, MsgSeq should be set to 0

If being sent by Receiver, MsgSeq should be set to next expected message number

MsgType 99 (Error)

KRDP 99 0000000000 0000 Error message here<CR>

Message number is 10 digit number 0000000000 to 2147483647

Message number indicates which message caused the error if any. Set to zero (0) if not related to a message number

0000 = Error number (0000 to 9999)

Error message can be any text

KRDP error messages

Error 1000 - Unable to decode the following message: <Invalid message appears here> A message was received that wasn't encoded correctly or corrupted. The message content appears for debugging purposes.

Error 1001 - Sender is unable to supply message number: <NextMsgSeq>. Starting again from 0. Sender ID: <UniqueSenderID> Expecting a sequence > 0, but sender unable to supply message, must start at 0 again. The receiver will now re-sync with the sender.

Error 1002 - Missed message number: <NextMsgSeq>. Received: <ActualMsgSeq> on ID: <UniqueSenderID> The expected message number was not received from the sender. The receiver will now re-sync with the sender.

Error 1003 - Received unexpected message data. Message ignored. Sender ID: <UniqueSenderID> Message data arrived while the receiver was not expecting it. This data is ignored.

Error 1004 - First message did not contain Sender ID. Connection closed. The first message received after connection was established did not contain the Sender ID. The receiver has closed the connection.

Error 1005 - Unable to send Expected message number reply. Connection closed. The receiver was unable to send a reply message over the established connection. The receiver has closed the connection.

Error 1006 - Unable to send error message. The receiver was unable to send an error message over the established connection.

Error 1007 - Unable to send KeepAlive message. Connection closed. The receiver was unable to send a KeepAlive message over the established connection. The receiver has closed the connection.

Error 1008 - Unable to send KeepAlive to connection: <UniqueSenderID> The receiver was unable to send a KeepAlive message over the established connection.

Error 1009 - Unable to send ACK to connection: <UniqueSenderID> The receiver was unable to send an ACK message over the established connection.

Error 1099 - <Error message content from sender> The sender can notify the receiver of an error by using the 1099 error type. The message content is from the sender.

Error 1010 - Unexpected message received. Type: <MsgType>. Message content: <Message Content> An unexpected message type was received. The message content appears for debugging purposes.