Comparison with NaN always returns true

[eliaslfox]

On macOS using CCL Version 1.12.1 DarwinX8664

Without float traps disabled comparing a value with NaN signals an error. With float traps disabled NaN values compare equal to themselves and other floats.

CL-USER> (ccl:set-fpu-mode :overflow nil :underflow nil :division-by-zero nil :invalid nil :inexact nil)
CL-USER> (= 1d+-0 1d+-0)
T
CL-USER> (= 1d+-0 1d)
T
CL-USER> (= 1d+-0 1f)
T

I looked at the disassembly for a function that compares floats:

(defun f (x y)
  (declare (type double-float x y)
           (optimize (speed 3) (safety 0)))
  (= x y))

On CCL COMISD is used to compare the arguments for equality. CCL then checks the equality flag (ZF) but does not check for unordered comparisons.

    (recover-fn-from-rip)                   ;     [0]
    (pushq (% rbp))                         ;     [7]
    (movq (% rsp) (% rbp))                  ;     [8]
    (pushq (% arg_y))                       ;    [11]
    (pushq (% arg_z))                       ;    [12]
    (movq (@ -16 (% rbp)) (% temp0))        ;    [13]
    (movsd (@ -5 (% temp0)) (% fp1))        ;    [17]
    (movq (@ -8 (% rbp)) (% temp0))         ;    [22]
    (movsd (@ -5 (% temp0)) (% fp0))        ;    [26]
    (comisd (% fp1) (% fp0))                ;    [31]
    (movl ($ #x1302E) (% imm0.l))           ;    [35]
    (movl ($ #x1300B) (% arg_z.l))          ;    [40]
    (cmovel (% imm0.l) (% arg_z.l))         ;    [45]
    (movq (% rbp) (% rsp))                  ;    [48]
    (popq (% rbp))                          ;    [51]
    (retq)                                  ;    [52]

For comparison, SBCL also uses COMISD but then it checks PF and ZF separately to handle unequal and unordered comparisons.

; disassembly for F
; Size: 34 bytes. Origin: #x5357E9B6                          ; F
; B6:       F20F1042F9       MOVSD XMM0, [RDX-7]

;;; [8] (= X Y)

; BB:       660F2F47F9       COMISD XMM0, [RDI-7]
; C0:       BA4F010050       MOV EDX, #x5000014F              ; T
; C5:       B817010050       MOV EAX, #x50000117              ; NIL
; CA:       480F4AD0         CMOVP RDX, RAX     ; <-- PF check here
; CE:       480F45D0         CMOVNE RDX, RAX

; D2:       488BE5           MOV RSP, RBP
; D5:       F8               CLC
; D6:       5D               POP RBP
; D7:       C3               RET

Also, CMOVE is called in CMOVEL throughout the CCL codebase. Is this intentional?

[xrme]

I agree that CCL needs to be smarter here. We'll have to introduce some new vinsn (or other mechanism) that can emit code to check multiple flags. The x8664 vinsn cr-bit->boolean only checks one flag.