Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ptpd hangs, OS UDP recv queue keeps growing #103

Open
apuri1 opened this issue Jul 18, 2023 · 4 comments
Open

ptpd hangs, OS UDP recv queue keeps growing #103

apuri1 opened this issue Jul 18, 2023 · 4 comments

Comments

@apuri1
Copy link

apuri1 commented Jul 18, 2023

We have a fleet of nvidia jetson xavier, arm64 computers, running ubuntu 20.04 with latest ptpd version installed.

>apt-cache policy ptpd
ptpd:
  Installed: 2.3.1-debian1-4
  Candidate: 2.3.1-debian1-4

>ptpd --version
ptpd2 version 2.3.1

We see random occurances whereby the UDP recv queue size starts and then keeps growing, where ptpd seems to not be processing the UDP packets anymore, and is using higher cpu than normal.

We see this from the node_exporter metrics, visualised:
image

top - 11:43:25 up 7 days, 21:45,  1 user,  load average: 7.12, 8.14, 7.98
Tasks:   1 total,   1 running,   0 sleeping,   0 stopped,   0 zombie
%Cpu(s): 31.6 us, 19.8 sy,  0.1 ni, 42.1 id,  0.0 wa,  4.2 hi,  2.1 si,  0.0 st
MiB Mem :  31011.1 total,  18653.7 free,   2087.4 used,  10270.1 buff/cache
MiB Swap:  15505.6 total,  15505.6 free,      0.0 used.  21450.9 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                            
3746034 root      20   0    3992   2384   1992 R  73.4   0.0 112:09.00 ptpd  

The logging seems to have stopped:

2023-07-17 19:04:41.906292 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Delay Response from Master
2023-07-18 02:03:49.890790 ptpd2[3746034].wlP4p1s0 (warning)   (slv) No active masters present. Resetting port.
2023-07-18 02:03:49.991663 ptpd2[3746034].wlP4p1s0 (notice)    (lstn_reset) Now in state: PTP_LISTENING
2023-07-18 02:03:51.880773 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Now in state: PTP_SLAVE, Best master: e45f01fffee4ed3b(unknown)/1
2023-07-18 02:03:51.890937 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Sync from Master
2023-07-18 02:03:52.897899 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Delay Response from Master
2023-07-18 08:40:33.875948 ptpd2[3746034].wlP4p1s0 (warning)   (slv) No active masters present. Resetting port.
2023-07-18 08:40:33.976929 ptpd2[3746034].wlP4p1s0 (notice)    (lstn_reset) Now in state: PTP_LISTENING
2023-07-18 08:40:35.877256 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Now in state: PTP_SLAVE, Best master: e45f01fffee4ed3b(unknown)/1
2023-07-18 08:40:37.877917 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Sync from Master
2023-07-18 08:40:38.884384 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Delay Response from Master

Running the strace command on the ptpd pid in this condition gives the output indicating a spinning condition with SIGALRM :

rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=6456, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=5837, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=2611, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=2528, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=1681, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=2221, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=2532, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---
rt_sigreturn({mask=[]})                 = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_TIMER, si_timerid=0x1, si_overrun=1861, si_value={int=-238370880, ptr=0xaaaaf1cabfc0}} ---

I then ran the linux perf utility on the same pid.

Screenshot for the bad case
image

compared to screenshot for a sample good case
image

The only remedy we have is to restart the ptpd service.

After restarting, we recover:

2023-07-18 08:40:35.877256 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Now in state: PTP_SLAVE, Best master: e45f01fffee4ed3b(unknown)/1
2023-07-18 08:40:37.877917 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Sync from Master
2023-07-18 08:40:38.884384 ptpd2[3746034].wlP4p1s0 (notice)    (slv) Received first Delay Response from Master


2023-07-18 12:01:35.620053 ptpd2[1375593].startup (info)      (___) Successfully bound ptpd2 to CPU core 0
2023-07-18 12:01:35.620757 ptpd2[1375593].startup (notice)    (___) PTPDv2 started successfully on wlP4p1s0 using "slaveonly" preset (PID 1375593)
2023-07-18 12:01:35.621233 ptpd2[1375593].startup (info)      (___) TimingService.PTP0: PTP service init
2023-07-18 12:01:35.632844 ptpd2[1375593].wlP4p1s0 (info)      (init) Observed drift loaded from /etc/ptpd2_kernelclock.drift: -32735 ppb
2023-07-18 12:01:35.733973 ptpd2[1375593].wlP4p1s0 (notice)    (lstn_init) Now in state: PTP_LISTENING
2023-07-18 12:01:37.928056 ptpd2[1375593].wlP4p1s0 (info)      (lstn_init) New best master selected: e45f01fffee4ed3b(unknown)/1
2023-07-18 12:01:37.928330 ptpd2[1375593].wlP4p1s0 (notice)    (slv) Now in state: PTP_SLAVE, Best master: e45f01fffee4ed3b(unknown)/1 (IPv4:10.228.4.100)
2023-07-18 12:01:37.928595 ptpd2[1375593].wlP4p1s0 (notice)    (slv) Received first Sync from Master
2023-07-18 12:01:37.929467 ptpd2[1375593].wlP4p1s0 (warning)   (slv) First clock update - will step the clock
2023-07-18 12:01:37.929605 ptpd2[1375593].wlP4p1s0 (warning)   (slv) First clock update and offset >= 1 second - will step the clock
2023-07-18 12:01:37.907700 ptpd2[1375593].wlP4p1s0 (warning)   (slv) Stepped the system clock to: 07/18/23 12:01:37.902493110
2023-07-18 12:01:37.931286 ptpd2[1375593].wlP4p1s0 (notice)    (lstn_reset) Now in state: PTP_LISTENING
2023-07-18 12:01:41.899855 ptpd2[1375593].wlP4p1s0 (info)      (lstn_reset) New best master selected: e45f01fffee4ed3b(unknown)/1
2023-07-18 12:01:41.900146 ptpd2[1375593].wlP4p1s0 (notice)    (slv) Now in state: PTP_SLAVE, Best master: e45f01fffee4ed3b(unknown)/1 (IPv4:10.228.4.100)
2023-07-18 12:01:44.896807 ptpd2[1375593].wlP4p1s0 (notice)    (slv) Received first Sync from Master
2023-07-18 12:01:45.594591 ptpd2[1375593].wlP4p1s0 (notice)    (slv) TimingService.PTP0: elected best TimingService
2023-07-18 12:01:45.594828 ptpd2[1375593].wlP4p1s0 (info)      (slv) TimingService.PTP0: acquired clock control
2023-07-18 12:01:50.594753 ptpd2[1375593].wlP4p1s0 (notice)    (slv) TimingService.PTP0: Major time change - syncing the RTC
2023-07-18 12:01:50.595757 ptpd2[1375593].wlP4p1s0 (notice)    (slv) Successfully set RTC time using /dev/rtc

and we see the UDP recv queue size drop
image

and the the new pid stable, running with expected low cpu usage.

short of constantly monitoring for this with alarms set to trigger on the growing UDP queue size and restarting the ptpd service, can this data be please analysed and a permanent solution implemented and released?

@apuri1
Copy link
Author

apuri1 commented Jul 18, 2023

The configuration file:

>cat /etc/ptpd/ptpd.client.conf 

; ========================================
; PTPDv2 version 2.3.1-rc5 default configuration
; ========================================

; NOTE: the following settings are affected by ptpengine:preset selection:
;           ptpengine:slave_only
;           clock:no_adjust
;           ptpengine:clock_class - allowed range and default value
; To see all preset settings, run ptpd2 -H (--long-help)

; Network interface to use - eth0, igb0 etc. (required).
ptpengine:interface = wlP4p1s0

; Backup network interface to use - eth0, igb0 etc. When no GM available, 
; slave will keep alternating between primary and secondary until a GM is found.
; 
ptpengine:backup_interface = 

; PTP engine preset:
; none	     = Defaults, no clock class restrictions
; masteronly  = Master, passive when not best master (clock class 0..127)
; masterslave = Full IEEE 1588 implementation:
; Master, slave when not best master
; (clock class 128..254)
; slaveonly   = Slave only (clock class 255 only)
; 
; Options: none masteronly masterslave slaveonly 
ptpengine:preset = slaveonly

; Transport type for PTP packets. Ethernet transport requires libpcap support.
; Options: ipv4 ethernet 
ptpengine:transport = ipv4

; Enable TransportSpecific field compatibility with 802.1AS / AVB (requires Ethernet transport)
ptpengine:dot2as = N

; IP transmission mode (requires IP transport) - hybrid mode uses
; multicast for sync and announce, and unicast for delay request and
; response; unicast mode uses unicast for all transmission.
; When unicast mode is selected, destination IP(s) may need to be configured
; (ptpengine:unicast_destinations).
; Options: multicast unicast hybrid 
ptpengine:ip_mode = hybrid

; Enable unicast negotiation support using signaling messages
; 
ptpengine:unicast_negotiation = N

; Disable Best Master Clock Algorithm for unicast masters:
; Only effective for masteronly preset - all Announce messages
; will be ignored and clock will transition directly into MASTER state.
; 
ptpengine:disable_bmca = N

; When unicast negotiation enabled on a master clock, 
; reply to transmission requests also in LISTENING state.
ptpengine:unicast_negotiation_listening = N

; Use libpcap for sending and receiving traffic (automatically enabled
; in Ethernet mode).
ptpengine:use_libpcap = N

; Delay detection mode used - use DELAY_DISABLED for syntonisation only
; (no full synchronisation).
; Options: E2E P2P DELAY_DISABLED 
ptpengine:delay_mechanism = E2E

; PTP domain number.
ptpengine:domain = 0

; PTP port number (part of PTP Port Identity - not UDP port).
; For ordinary clocks (single port), the default should be used, 
; but when running multiple instances to simulate a boundary clock, 
; The port number can be changed.
ptpengine:port_number = 1

; Usability extension: if enabled, a slave-only clock will accept
; masters from any domain, while preferring the configured domain,
; and preferring lower domain number.
; NOTE: this behaviour is not part of the standard.
ptpengine:any_domain = N

; Slave only mode (sets clock class to 255, overriding value from preset).
ptpengine:slave_only = Y

; Specify latency correction (nanoseconds) for incoming packets.
ptpengine:inbound_latency = 0

; Specify latency correction (nanoseconds) for outgoing packets.
ptpengine:outbound_latency = 0

; Apply an arbitrary shift (nanoseconds) to offset from master when
; in slave state. Value can be positive or negative - useful for
; correcting for antenna latencies, delay assymetry
; and IP stack latencies. This will not be visible in the offset 
; from master value - only in the resulting clock correction.
ptpengine:offset_shift = 0

; Compatibility option: In slave state, always respect UTC offset
; announced by best master, even if the the
; currrentUtcOffsetValid flag is announced FALSE.
; NOTE: this behaviour is not part of the standard.
ptpengine:always_respect_utc_offset = N

; Compatibility extension to BMC algorithm: when enabled,
; BMC for both master and save clocks will prefer masters
; nannouncing currrentUtcOffsetValid as TRUE.
; NOTE: this behaviour is not part of the standard.
ptpengine:prefer_utc_offset_valid = N

; Compatibility option: when enabled, ptpd2 will ignore
; Announce messages from masters announcing currentUtcOffsetValid
; as FALSE.
; NOTE: this behaviour is not part of the standard.
ptpengine:require_utc_offset_valid = N

; Time (seconds) unicast messages are requested for by slaves
; when using unicast negotiation, and maximum time unicast message
; transmission is granted to slaves by masters
; 
ptpengine:unicast_grant_duration = 300

; PTP announce message interval in master state. When using unicast negotiation, for
; slaves this is the minimum interval requested, and for masters
; this is the only interval granted.
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_announce_interval = 1

; Maximum Announce message interval requested by slaves when using unicast negotiation,
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_announce_interval_max = 5

; PTP announce receipt timeout announced in master state.
ptpengine:announce_receipt_timeout = 6

; PTP announce receipt timeout grace period in slave state:
; when announce receipt timeout occurs, disqualify current best GM,
; then wait n times announce receipt timeout before resetting.
; Allows for a seamless GM failover when standby GMs are slow
; to react. When set to 0, this option is not used.
ptpengine:announce_receipt_grace_period = 0

; PTP sync message interval in master state. When using unicast negotiation, for
; slaves this is the minimum interval requested, and for masters
; this is the only interval granted.
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_sync_interval = 0

; Maximum Sync message interval requested by slaves when using unicast negotiation,
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_sync_interval_max = 5

; Override the Delay Request interval announced by best master.
ptpengine:log_delayreq_override = N

; Automatically override the Delay Request interval
; if the announced value is 127 (0X7F), such as in
; unicast messages (unless using unicast negotiation)
ptpengine:log_delayreq_auto = Y

; Delay request interval used before receiving first delay response
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_delayreq_interval_initial = 0

; Minimum delay request interval announced when in master state,
; in slave state overrides the master interval,
; required in hybrid mode. When using unicast negotiation, for
; slaves this is the minimum interval requested, and for masters
; this is the minimum interval granted.
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_delayreq_interval = 0

; Maximum Delay Response interval requested by slaves when using unicast negotiation,
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_delayreq_interval_max = 5

; Minimum peer delay request message interval in peer to peer delay mode.
; When using unicast negotiation, this is the minimum interval requested, 
; and the only interval granted.
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_peer_delayreq_interval = 1

; Maximum Peer Delay Response interval requested by slaves when using unicast negotiation,
; (expressed as log 2 i.e. -1=0.5s, 0=1s, 1=2s etc.)
ptpengine:log_peer_delayreq_interval_max = 5

; Foreign master record size (Maximum number of foreign masters).
ptpengine:foreignrecord_capacity = 5

; Specify Allan variance announced in master state.
ptpengine:ptp_allan_variance = 65535

; Clock accuracy range announced in master state.
; Options: ACC_25NS ACC_100NS ACC_250NS ACC_1US ACC_2.5US ACC_10US ACC_25US ACC_100US ACC_250US ACC_1MS ACC_2.5MS ACC_10MS ACC_25MS ACC_100MS ACC_250MS ACC_1S ACC_10S ACC_10SPLUS ACC_UNKNOWN 
ptpengine:ptp_clock_accuracy = ACC_UNKNOWN

; Underlying time source UTC offset announced in master state.
ptpengine:utc_offset = 0

; Underlying time source UTC offset validity announced in master state.
ptpengine:utc_offset_valid = N

; Underlying time source time traceability announced in master state.
ptpengine:time_traceable = N

; Underlying time source frequency traceability announced in master state.
ptpengine:frequency_traceable = N

; Time scale announced in master state (with ARB, UTC properties
; are ignored by slaves). When clock class is set to 13 (application
; specific), this value is ignored and ARB is used.
; Options: PTP ARB 
ptpengine:ptp_timescale = PTP

; Time source announced in master state.
; Options: ATOMIC_CLOCK GPS TERRESTRIAL_RADIO PTP NTP HAND_SET OTHER INTERNAL_OSCILLATOR 
ptpengine:ptp_timesource = INTERNAL_OSCILLATOR

; Clock class - announced in master state. Always 255 for slave-only.
; Minimum, maximum and default values are controlled by presets.
; If set to 13 (application specific time source), announced 
; time scale is always set to ARB. This setting controls the
; states a PTP port can be in. If below 128, port will only
; be in MASTER or PASSIVE states (master only). If above 127,
; port will be in MASTER or SLAVE states.
ptpengine:clock_class = 255

; Priority 1 announced in master state,used for Best Master
; Clock selection.
ptpengine:priority1 = 128

; Priority 2 announced in master state, used for Best Master
; Clock selection.
ptpengine:priority2 = 128

; Number of consecutive resets to LISTENING before full network reset
; 
ptpengine:max_listen = 5

; Specify unicast slave addresses for unicast master operation, or unicast
; master addresses for slave operation. Format is similar to an ACL: comma,
; tab or space-separated IPv4 unicast addresses, one or more. For a slave,
; when unicast negotiation is used, setting this is mandatory.
ptpengine:unicast_destinations = 

; Specify PTP domain number for each configured unicast destination (ptpengine:unicast_destinations).
; This is only used by slave-only clocks using unicast destinations to allow for each master
; to be in a separate domain, such as with Telecom Profile. The number of entries should match the number
; of unicast destinations, otherwise unconfigured domains or domains set to 0 are set to domain configured in
; ptpengine:domain. The format is a comma, tab or space-separated list of 8-bit unsigned integers (0 .. 255)
ptpengine:unicast_domains = 

; Specify a local preference for each configured unicast destination (ptpengine:unicast_destinations).
; This is only used by slave-only clocks using unicast destinations to allow for each master's
; BMC selection to be influenced by the slave, such as with Telecom Profile. The number of entries should match the number
; of unicast destinations, otherwise unconfigured preference is set to 0 (highest).
; The format is a comma, tab or space-separated list of 8-bit unsigned integers (0 .. 255)
ptpengine:unicast_local_preference = 

; Specify peer unicast adress for P2P unicast. Mandatory when
; running unicast mode and P2P delay mode.
ptpengine:unicast_peer_destination = 

; Enable handling of PTP management messages.
ptpengine:management_enable = Y

; Accept SET and COMMAND management messages.
ptpengine:management_set_enable = N

; Send explicit IGMP joins between engine resets and periodically
; in master state.
ptpengine:igmp_refresh = Y

; Periodic IGMP join interval (seconds) in master state when running
; IPv4 multicast: when set below 10 or when ptpengine:igmp_refresh
; is disabled, this setting has no effect.
ptpengine:master_igmp_refresh_interval = 60

; Multicast time to live for multicast PTP packets (ignored and set to 1
; for peer to peer messages).
ptpengine:multicast_ttl = 64

; DiffServ CodepPoint for packet prioritisation (decimal). When set to zero, 
; this option is not used. Use 46 for Expedited Forwarding (0x2e).
ptpengine:ip_dscp = 46

; Enable statistical filter for Sync messages.
ptpengine:sync_stat_filter_enable = Y

; Type of filter used for Sync message filtering
; Options: none mean min max absmin absmax median 
ptpengine:sync_stat_filter_type = min

; Number of samples used for the Sync statistical filter
ptpengine:sync_stat_filter_window = 16

; Sample window type used for Sync message statistical filter. Delay Response outlier filter action.
; Sliding window is continuous, interval passes every n-th sample only.
; Options: sliding interval 
ptpengine:sync_stat_filter_window_type = sliding

; Enable statistical filter for Delay messages.
ptpengine:delay_stat_filter_enable = Y

; Type of filter used for Delay message statistical filter
; Options: none mean min max absmin absmax median 
ptpengine:delay_stat_filter_type = min

; Number of samples used for the Delay statistical filter
ptpengine:delay_stat_filter_window = 16

; Sample window type used for Delay message statistical filter
; Sliding window is continuous, interval passes every n-th sample only
; Options: sliding interval 
ptpengine:delay_stat_filter_window_type = sliding

; Enable outlier filter for the Delay Response component in slave state
ptpengine:delay_outlier_filter_enable = N

; Delay Response outlier filter action. If set to 'filter', outliers are
; replaced with moving average.
; Options: discard filter 
ptpengine:delay_outlier_filter_action = discard

; Number of samples in the Delay Response outlier filter buffer
ptpengine:delay_outlier_filter_capacity = 20

; Delay Response outlier filter threshold (: multiplier for Peirce's maximum
; standard deviation. When set below 1.0, filter is tighter, when set above
; 1.0, filter is looser than standard Peirce's test.
; When autotune enabled, this is the starting threshold.
ptpengine:delay_outlier_filter_threshold = 1.000000

; Always run the Delay Response outlier filter, even if clock is being slewed at maximum rate
ptpengine:delay_outlier_filter_always_filter = N

; Enable automatic threshold control for Delay Response outlier filter.
ptpengine:delay_outlier_filter_autotune_enable = Y

; Delay Response outlier filter autotune low watermark - minimum percentage
; of discarded samples in the update period before filter is tightened
; by the autotune step value.
ptpengine:delay_outlier_filter_autotune_minpercent = 20

; Delay Response outlier filter autotune high watermark - maximum percentage
; of discarded samples in the update period before filter is loosened
; by the autotune step value.
ptpengine:delay_outlier_filter_autotune_maxpercent = 95

; The value the Delay Response outlier filter threshold is increased
; or decreased by when auto-tuning.
ptpengine:delay_outlier_autotune_step = 0.100000

; Minimum Delay Response filter threshold value used when auto-tuning
ptpengine:delay_outlier_filter_autotune_minthreshold = 0.100000

; Maximum Delay Response filter threshold value used when auto-tuning
ptpengine:delay_outlier_filter_autotune_maxthreshold = 5.000000

; Enable Delay filter step detection (delaySM) to block when certain level exceeded
ptpengine:delay_outlier_filter_stepdetect_enable = N

; Delay Response step detection threshold. Step detection is performed
; only when delaySM is below this threshold (nanoseconds)
ptpengine:delay_outlier_filter_stepdetect_threshold = 1000000

; Delay Response step level. When step detection enabled and operational,
; delaySM above this level (nanosecond) is considered a clock step and updates are paused
ptpengine:delay_outlier_filter_stepdetect_level = 500000

; Initial credit (number of samples) the Delay step detection filter can block for
; When credit is exhausted, filter stops blocking. Credit is gradually restored
ptpengine:delay_outlier_filter_stepdetect_credit = 200

; Amount of credit for the Delay step detection filter restored every full sample window
ptpengine:delay_outlier_filter_stepdetect_credit_increment = 10

; Delay Response outlier weight: if an outlier is detected, determines
; the amount of its deviation from mean that is used to build the standard
; deviation statistics and influence further outlier detection.
; When set to 1.0, the outlier is used as is.
ptpengine:delay_outlier_weight = 1.000000

; Enable outlier filter for the Sync component in slave state.
ptpengine:sync_outlier_filter_enable = N

; Sync outlier filter action. If set to 'filter', outliers are replaced
; with moving average.
; Options: discard filter 
ptpengine:sync_outlier_filter_action = discard

; Number of samples in the Sync outlier filter buffer.
ptpengine:sync_outlier_filter_capacity = 20

; Sync outlier filter threshold: multiplier for the Peirce's maximum standard
; deviation. When set below 1.0, filter is tighter, when set above 1.0,
; filter is looser than standard Peirce's test.
ptpengine:sync_outlier_filter_threshold = 1.000000

; Always run the Sync outlier filter, even if clock is being slewed at maximum rate
ptpengine:sync_outlier_filter_always_filter = N

; Enable automatic threshold control for Sync outlier filter.
ptpengine:sync_outlier_filter_autotune_enable = Y

; Sync outlier filter autotune low watermark - minimum percentage
; of discarded samples in the update period before filter is tightened
; by the autotune step value.
ptpengine:sync_outlier_filter_autotune_minpercent = 20

; Sync outlier filter autotune high watermark - maximum percentage
; of discarded samples in the update period before filter is loosened
; by the autotune step value.
ptpengine:sync_outlier_filter_autotune_maxpercent = 95

; Value the Sync outlier filter threshold is increased
; or decreased by when auto-tuning.
ptpengine:sync_outlier_autotune_step = 0.100000

; Minimum Sync outlier filter threshold value used when auto-tuning
ptpengine:sync_outlier_filter_autotune_minthreshold = 0.100000

; Maximum Sync outlier filter threshold value used when auto-tuning
ptpengine:sync_outlier_filter_autotune_maxthreshold = 5.000000

; Enable Sync filter step detection (delayMS) to block when certain level exceeded.
ptpengine:sync_outlier_filter_stepdetect_enable = N

; Sync step detection threshold. Step detection is performed
; only when delayMS is below this threshold (nanoseconds)
ptpengine:sync_outlier_filter_stepdetect_threshold = 1000000

; Sync step level. When step detection enabled and operational,
; delayMS above this level (nanosecond) is considered a clock step and updates are paused
ptpengine:sync_outlier_filter_stepdetect_level = 500000

; Initial credit (number of samples) the Sync step detection filter can block for.
; When credit is exhausted, filter stops blocking. Credit is gradually restored
ptpengine:sync_outlier_filter_stepdetect_credit = 200

; Amount of credit for the Sync step detection filter restored every full sample window
ptpengine:sync_outlier_filter_stepdetect_credit_increment = 10

; Sync outlier weight: if an outlier is detected, this value determines the
; amount of its deviation from mean that is used to build the standard 
; deviation statistics and influence further outlier detection.
; When set to 1.0, the outlier is used as is.
ptpengine:sync_outlier_weight = 1.000000

; Delay between moving to slave state and enabling clock updates (seconds).
; This allows one-way delay to stabilise before starting clock updates.
; Activated when going into slave state and during slave's GM failover.
; 0 - not used.
ptpengine:calibration_delay = 0

; PTP idle timeout: if PTPd is in SLAVE state and there have been no clock
; updates for this amout of time, PTPd releases clock control.
; 
ptpengine:idle_timeout = 120

; Enable panic mode: when offset from master is above 1 second, stop updating
; the clock for a period of time and then step the clock if offset remains
; above 1 second.
ptpengine:panic_mode = N

; Duration (minutes) of the panic mode period (no clock updates) when offset
; above 1 second detected.
ptpengine:panic_mode_duration = 2

; When entering panic mode, release clock control while panic mode lasts
; if ntpengine:* configured, this will fail over to NTP,
; if not set, PTP will hold clock control during panic mode.
ptpengine:panic_mode_release_clock = N

; Do not exit panic mode until offset drops below this value (nanoseconds).
; 0 = not used.
ptpengine:panic_mode_exit_threshold = 0

; Use PTPd's process ID as the middle part of the PTP clock ID - useful for running multiple instances.
ptpengine:pid_as_clock_identity = N

; Fail over to NTP when PTP time sync not available - requires
; ntpengine:enabled, but does not require the rest of NTP configuration:
; will warn instead of failing over if cannot control ntpd.
ptpengine:ntp_failover = N

; NTP failover timeout in seconds: time between PTP slave going into
; LISTENING state, and releasing clock control. 0 = fail over immediately.
ptpengine:ntp_failover_timeout = 120

; Prefer NTP time synchronisation. Only use PTP when NTP not available,
; could be used when NTP runs with a local GPS receiver or another reference
ptpengine:prefer_ntp = N

; Legacy option from 2.3.0: same as ptpengine:panic_mode_release_clock
ptpengine:panic_mode_ntp = N

; Clear counters after dumping all counter values on SIGUSR2.
ptpengine:sigusr2_clears_counters = N

; Permit access control list for timing packets. Format is a series of 
; comma, space or tab separated  network prefixes: IPv4 addresses or full CIDR notation a.b.c.d/x,
; where a.b.c.d is the subnet and x is the decimal mask, or a.b.c.d/v.x.y.z where a.b.c.d is the
; subnet and v.x.y.z is the 4-octet mask. The match is performed on the source IP address of the
; incoming messages. IP access lists are only supported when using the IP transport.
ptpengine:timing_acl_permit = 

; Deny access control list for timing packets. Format is a series of 
; comma, space or tab separated  network prefixes: IPv4 addresses or full CIDR notation a.b.c.d/x,
; where a.b.c.d is the subnet and x is the decimal mask, or a.b.c.d/v.x.y.z where a.b.c.d is the
; subnet and v.x.y.z is the 4-octet mask. The match is performed on the source IP address of the
; incoming messages. IP access lists are only supported when using the IP transport.
ptpengine:timing_acl_deny = 

; Permit access control list for management messages. Format is a series of 
; comma, space or tab separated  network prefixes: IPv4 addresses or full CIDR notation a.b.c.d/x,
; where a.b.c.d is the subnet and x is the decimal mask, or a.b.c.d/v.x.y.z where a.b.c.d is the
; subnet and v.x.y.z is the 4-octet mask. The match is performed on the source IP address of the
; incoming messages. IP access lists are only supported when using the IP transport.
ptpengine:management_acl_permit = 

; Deny access control list for management messages. Format is a series of 
; comma, space or tab separated  network prefixes: IPv4 addresses or full CIDR notation a.b.c.d/x,
; where a.b.c.d is the subnet and x is the decimal mask, or a.b.c.d/v.x.y.z where a.b.c.d is the
; subnet and v.x.y.z is the 4-octet mask. The match is performed on the source IP address of the
; incoming messages. IP access lists are only supported when using the IP transport.
ptpengine:management_acl_deny = 

; Order in which permit and deny access lists are evaluated for timing
; packets, the evaluation process is the same as for Apache httpd.
; Options: permit-deny deny-permit 
ptpengine:timing_acl_order = deny-permit

; Order in which permit and deny access lists are evaluated for management
; messages, the evaluation process is the same as for Apache httpd.
; Options: permit-deny deny-permit 
ptpengine:management_acl_order = deny-permit

; Do not adjust the clock
clock:no_adjust = N

; Do not step the clock - only slew
clock:no_reset = N

; Force clock step on first sync after startup regardless of offset and clock:no_reset
clock:step_startup_force = Y

; Step clock on startup if offset >= 1 second, ignoring
; panic mode and clock:no_reset
clock:step_startup = Y

; Attempt setting the RTC when stepping clock (Linux only - FreeBSD does 
; this for us. WARNING: this will always set the RTC to OS clock time,
; regardless of time zones, so this assumes that RTC runs in UTC or 
; at least in the same timescale as PTP. true at least on most 
; single-boot x86 Linux systems.
clock:set_rtc_on_step = Y

; Observed drift handling method between servo restarts:
; reset: set to zero (not recommended)
; preserve: use kernel value,
; file: load/save to drift file on startup/shutdown, use kernel
; value inbetween. To specify drift file, use the clock:drift_file setting.
; Options: reset preserve file 
clock:drift_handling = file

; Specify drift file
clock:drift_file = /etc/ptpd2_kernelclock.drift

; Time (seconds) before and after midnight that clock updates should pe suspended for
; during a leap second event. The total duration of the pause is twice
; the configured duration
clock:leap_second_pause_period = 5

; Time (seconds) before midnight that PTPd starts announcing the leap second
; if it's running as master
clock:leap_second_notice_period = 43200

; Specify leap second file location - up to date version can be downloaded from 
; http://www.ietf.org/timezones/data/leap-seconds.list
clock:leap_seconds_file = 

; Behaviour during a leap second event:
; accept: inform the OS kernel of the event
; ignore: do nothing - ends up with a 1-second offset which is then slewed
; step: similar to ignore, but steps the clock immediately after the leap second event
; smear: do not inform kernel, gradually introduce the leap second before the event
; by modifying clock offset (see clock:leap_second_smear_period)
; Options: accept ignore step smear 
clock:leap_second_handling = accept

; Time period (Seconds) over which the leap second is introduced before the event.
; Example: when set to 86400 (24 hours), an extra 11.5 microseconds is added every second
clock:leap_second_smear_period = 86400

; Maximum absolute frequency shift which can be applied to the clock servo
; when slewing the clock. Expressed in parts per million (1 ppm = shift of
; 1 us per second. Values above 512 will use the tick duration correction
; to allow even faster slewing. Default maximum is 512 without using tick.
clock:max_offset_ppm = 1000

; One-way delay filter stiffness.
servo:delayfilter_stiffness = 6

; Clock servo PI controller proportional component gain (kP).
servo:kp = 0.100000

; Clock servo PI controller integral component gain (kI).
servo:ki = 0.001000

; How servo update interval (delta t) is calculated:
; none:     servo not corrected for update interval (dt always 1),
; constant: constant value (target servo update rate - sync interval for PTP,
; measured: servo measures how often it's updated and uses this interval.
; Options: none constant measured 
servo:dt_method = constant

; Maximum servo update interval (delta t) when using measured servo update interval
; (servo:dt_method = measured), specified as sync interval multiplier.
servo:dt_max = 5.000000

; Enable clock synchronisation servo stability detection
; (based on standard deviation of the observed drift value)
; - drift will be saved to drift file / cached when considered stable,
; also clock stability status will be logged.
servo:stability_detection = N

; Specify the observed drift standard deviation threshold in parts per
; billion (ppb) - if stanard deviation is within the threshold, servo
; is considered stable.
servo:stability_threshold = 10.000000

; Specify for how many statistics update intervals the observed drift
; standard deviation has to stay within threshold to be considered stable.
servo:stability_period = 1

; Specify after how many minutes without stabilisation servo is considered
; unstable. Assists with logging servo stability information and
; allows to preserve observed drift if servo cannot stabilise.
; 
servo:stability_timeout = 10

; Do accept master to slave delay (delayMS - from Sync message) or slave to master delay
; (delaySM - from Delay messages) if greater than this value (nanoseconds). 0 = not used.
servo:max_delay = 0

; Maximum number of consecutive delay measurements exceeding maxDelay threshold,
; before slave is reset.
servo:max_delay_max_rejected = 0

; If servo:max_delay is set, perform the check only if clock servo has stabilised.
; 
servo:max_delay_stable_only = N

; When enabled, Sync messages will only be accepted if sequence ID is increasing.        This is limited to 50 dropped messages.
; 
ptpengine:sync_sequence_checking = N

; If set to non-zero, timeout in seconds, after which the slave resets if no clock updates made. 
; 
ptpengine:clock_update_timeout = 0

; Do not reset the clock if offset from master is greater
; than this value (nanoseconds). 0 = not used.
servo:max_offset = 0

; Enable SNMP agent (if compiled with PTPD_SNMP).
;global:enable_snmp = N

; Send log messages to syslog. Disabling this
; sends all messages to stdout (or speficied log file).
global:use_syslog = N

; Lock file location
global:lock_file = 

; 	 Use mode specific and interface specific lock file
; (overrides global:lock_file).
global:auto_lockfile = N

; Lock file directory: used with automatic mode-specific lock files,
; also used when no lock file is specified. When lock file
; is specified, it's expected to be an absolute path.
global:lock_directory = /var/run

; Skip lock file checking and locking.
global:ignore_lock = N

; File used to record data about sync packets. Enables recording when set.
global:quality_file = 

; Maximum sync packet record file size (in kB) - file will be truncated
; if size exceeds the limit. 0 - no limit.
global:quality_file_max_size = 0

; Enable log rotation of the sync packet record file up to n files.
; 0 - do not rotate.
; 
global:quality_file_max_files = 0

; Truncate the sync packet record file every time it is (re) opened:
; startup and SIGHUP.
global:quality_file_truncate = N

; File used to log ptpd2 status information.
global:status_file = /var/run/ptpd2.status

; Enable / disable writing status information to file.
global:log_status = N

; Status file update interval in seconds.
global:status_update_interval = 1

; Specify log file path (event log). Setting this enables logging to file.
global:log_file = /var/log/ptpd2.log

; Maximum log file size (in kB) - log file will be truncated if size exceeds
; the limit. 0 - no limit.
global:log_file_max_size = 5000

; Enable log rotation of the sync packet record file up to n files.
; 0 - do not rotate.
; 
global:log_file_max_files = 5

; Truncate the log file every time it is (re) opened: startup and SIGHUP.
global:log_file_truncate = N

; Specify log level (only messages at this priority or higer will be logged).
; The minimal level is LOG_ERR. LOG_ALL enables debug output if compiled with
; RUNTIME_DEBUG.
; Options: LOG_ERR LOG_WARNING LOG_NOTICE LOG_INFO LOG_ALL 
global:log_level = LOG_ALL

; Specify statistics log file path. Setting this enables logging of 
; statistics, but can be overriden with global:log_statistics.
global:statistics_file = /var/log/ptpd2.stats

; Log timing statistics every n seconds for Sync and Delay messages
; (0 - log all).
global:statistics_log_interval = 0

; Maximum statistics log file size (in kB) - log file will be truncated
; if size exceeds the limit. 0 - no limit.
global:statistics_file_max_size = 0

; Enable log rotation of the statistics file up to n files. 0 - do not rotate.
global:statistics_file_max_files = 0

; Truncate the statistics file every time it is (re) opened: startup and SIGHUP.
global:statistics_file_truncate = N

; Dump the contents of every PTP packet
global:dump_packets = N

; Run in foreground with statistics and all messages logged to stdout.
; Overrides log file and statistics file settings and disables syslog.
; 
global:verbose_foreground = N

; Run in foreground - ignored when global:verbose_foreground is set
global:foreground = Y

; Log timing statistics for every PTP packet received
; 
global:log_statistics = Y

; Timestamp format used when logging timing statistics
; (when global:log_statistics is enabled):
; datetime - formatttted date and time: YYYY-MM-DD hh:mm:ss.uuuuuu
; unix - Unix timestamp with nanoseconds: s.ns
; both - Formatted date and time, followed by unix timestamp
; (adds one extra field  to the log)
; 
; Options: datetime unix both 
global:statistics_timestamp_format = datetime

; Bind ptpd2 process to a selected CPU core number.
; 0 = first CPU core, etc. -1 = do not bind to a single core.
global:cpuaffinity_cpucore = 0

; Clock synchronisation statistics update interval in seconds
; 
global:statistics_update_interval = 30

; Log a status update every time statistics are updated (global:statistics_update_interval).
; The updates are logged even when ptpd is configured without statistics support
global:periodic_updates = N

;  Delay (seconds) before releasing a time service (NTP or PTP)        and electing a new one to control a clock. 0 = elect immediately
; 
global:timingdomain_election_delay = 15

; Enable NTPd integration
ntpengine:enabled = N

; Enable control over local NTPd daemon
ntpengine:control_enabled = N

; NTP control check interval in seconds
; 
ntpengine:check_interval = 15

; NTP key number - must be configured as a trusted control key in ntp.conf,
; and be non-zero for the ntpengine:control_enabled setting to take effect.
; 
ntpengine:key_id = 0

; NTP key (plain text, max. 20 characters) - must match the key configured in
; ntpd's keys file, and must be non-zero for the ntpengine:control_enabled
; setting to take effect.
; 
ntpengine:key = 

; ========= newline required in the end ==========

@FPSychotic
Copy link

hi sir, It looks I could be running in the same issue, but I have no knowledge to can be sure. What I know at least is the doemon won't run at boot , and the service status will send the error "Interface eth0 has no IPv4 address set"
I have a Orin NX with jetpack 5.1.2. In the same way if I can support you I will happy to make any test that help you.

thanks.

@apuri1
Copy link
Author

apuri1 commented Sep 12, 2023

The fix is here: https://github.com/ptpd/ptpd/blob/master/ChangeLog#L24

- critical: added minimum POSIX timer interval to prevent from
  timers firing to quickly for the process to handle,
  resulting in 100% CPU and endless signal queue

2.3.2 is not a release tag, so need to pull & build latest from master.

@apuri1
Copy link
Author

apuri1 commented Sep 12, 2023

Will keep this ticket open until the latest mods are released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants