Windows - Can't run test script + lots of failing mocha tests

[StarpTech]

My environment:
Node 6.9.2
Windows 10
gnatsd-v0.9.4-windows-amd64

Im trying to run the test but Im facing with many issues

1. Seems that the command is not optimitzed for windows

E:\Repositorys\node-nats>npm run test

> nats@0.7.0 test E:\Repositorys\node-nats
> npm run depcheck && npm run depcheck:unused && npm run lint && npm run test:unit


> nats@0.7.0 depcheck E:\Repositorys\node-nats
> dependency-check . lib/*

Success! All dependencies used in the code are listed in package.json

> nats@0.7.0 depcheck:unused E:\Repositorys\node-nats
> dependency-check ./package.json --unused --no-dev lib/*

Success! All dependencies in package.json are used in the code

> nats@0.7.0 lint E:\Repositorys\node-nats
> jshint --reporter node_modules/jshint-stylish lib/*.js test/*.js test/support/*.js examples/*.js

ERROR: Can't open lib/*.js
ERROR: Can't open test/*.js
ERROR: Can't open test/support/*.js
ERROR: Can't open examples/*.js

√ No problems


> nats@0.7.0 test:unit E:\Repositorys\node-nats
> mkdir -p reports/ && NODE_ENV=test multi='spec=- xunit=reports/mocha-xunit.xml' istanbul cover _mocha -- -R mocha-multi --timeout 10000 --slow 750 && istanbul check-coverage

The syntax of the command is incorrect.

2. I skip that above and try mocha --timeout 10000 and I have 25 failing tests.

 mocha --timeout 10000


  Authorization
    1) should fail to connect with no credentials
    √ should connect with proper credentials in url (102ms)
    √ should connect with proper credentials as options (102ms)
    √ should connect with proper credentials as server url (103ms)

  Token Authorization
    √ should fail to connect with no credentials
    √ should connect with proper credentials in url (102ms)
    √ should connect with proper credentials as options (101ms)
    √ should connect with proper credentials as server url (101ms)

  Max responses and Auto-unsub
    √ should only received max responses requested
    √ should only received max responses requested (client support)
    √ should not complain when unsubscribing an auto-unsubscribed sid
    √ should allow proper override to a lesser value
    √ should allow proper override to a higher value
    √ should only receive N msgs in request mode with multiple helpers
    √ should not leak subscriptions when using max (102ms)

  Basics
    √ should do basic subscribe and unsubscribe
    √ should do basic publish
    √ should fire a callback for subscription
    √ should include the correct message in the callback
    √ should include the correct reply in the callback
    √ should do request-reply
    √ should return a sub id for requests
    √ should do single partial wildcard subscriptions correctly
    √ should do partial wildcard subscriptions correctly
    √ should do full wildcard subscriptions correctly
    √ should pass exact subject to callback
    √ should do callback after publish is flushed
    √ should do callback after flush
    √ should handle an unsubscribe after close of connection
    √ should not receive data after unsubscribe call
    √ should pass sid properly to a message callback if requested
    √ should parse json messages
    √ should parse UTF8 json messages
    √ should validate json messages before publishing

  Binary
    √ should allow sending and receiving binary data

  Callbacks
    √ should properly do a publish callback after connection is closed
    √ should properly do a flush callback after connection is closed

  Cluster
    √ should accept servers options
    √ should randomly connect to servers by default
    √ should connect to first valid server
    √ should emit error if no servers are available (2007ms)
    √ should not randomly connect to servers if noRandomize is set
    √ should not randomly connect to servers if dontRandomize is set
    2) should fail after maxReconnectAttempts when servers killed

  Basic Connectivity
    √ should perform basic connect with port
    √ should perform basic connect with uri
    √ should perform basic connect with options arg
    √ should emit a connect event
    3) should emit error if no server available
    √ should emit connecting events and try repeatedly if configured and no server available
    4) should still receive publish when some servers are invalid
    5) should still receive publish when some servers[noRandomize] are invalid
    √ should add a new cluster server

  Double SUBS
    6) "before all" hook

  Errors
    √ should throw errors on connect
    √ should throw errors on publish
    √ should throw errors on flush
    √ should pass errors on publish with callbacks
    √ should throw errors on subscribe
    √ NatsErrors have code
    √ NatsErrors can chain an error

  JSON payloads
    7) "before all" hook

  Base Properties
    √ should have a version property
    √ should have the same version as package.json
    √ should have a connect function
    √ should have a createInbox function

  Connection Properties
    √ should have a publish function
    √ should have a subscribe function
    √ should have an unsubscribe function
    √ should have a request function
    √ should have an options hash with proper fields
    √ should have an parsed url
    √ should allow options to be overridden

  Queues
    √ should deliver a message to single member of a queue group
    √ should deliver a message to only one member of a queue group
    √ should allow queue subscribers and normal subscribers to work together
    √ should spread messages out equally (given random)
    √ should deliver only one mesage to queue subscriber regardless of wildcards
    √ should deliver to multiple queue groups

  Reconnect functionality
    √ should not emit a reconnecting event if suppressed (254ms)
    8) should emit a disconnect and a reconnecting event after proper delay
    9) should emit multiple reconnecting events and fail after maxReconnectAttempts
    10) should emit reconnecting events indefinitely if maxReconnectAttempts is set to -1
    11) should succesfully reconnect to new server
    12) should succesfully reconnect to new server with subscriptions
    13) should succesfully reconnect to new server with queue subscriptions correctly
    14) should properly resync with inbound buffer non-nil
    15) should not crash when sending a publish with a callback after connection loss
    16) should execute callbacks if published during reconnect
    17) should not lose messages if published during reconnect
    18) should emit reconnect before flush callbacks are called

  Split Messages
    19) "before all" hook

  Subscription Events
    20) "before all" hook

  Timeout and max received events for subscriptions
    21) "before all" hook

  TLS
    22) "before all" hook
    23) "after all" hook

  UTF8
    24) "before all" hook
    Yield
      25) "before all" hook


  73 passing (13s)
  25 failing

Errors:

1) Authorization should fail to connect with no credentials :
     Uncaught AssertionError: expected null to exist
      at Client.<anonymous> (E:\Repositorys\node-nats\test\auth.js:31:14)
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:14)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  2) Cluster should fail after maxReconnectAttempts when servers killed:
     Uncaught AssertionError: expected 1024 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\cluster.js:144:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  3) Basic Connectivity should emit error if no server available:
     Uncaught AssertionError: expected 2049 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\cluster.js:144:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  4) Basic Connectivity should still receive publish when some servers are invalid:

      Uncaught AssertionError: expected '' to be 'hello'
      + expected - actual

      +hello

      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Timeout._onTimeout (E:\Repositorys\node-nats\test\connect.js:94:22)

  5) Basic Connectivity should still receive publish when some servers[noRandomize] are invalid:

      Uncaught AssertionError: expected 0 to be 5
      + expected - actual

      -0
      +5

      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Timeout._onTimeout (E:\Repositorys\node-nats\test\connect.js:76:31)

  6) Double SUBS "before all" hook:
     Error: Can't connect to server on port: 1922
      at Timeout.<anonymous> (E:\Repositorys\node-nats\test\support\nats_server_control.js:58:14)

  7) JSON payloads "before all" hook:
     Uncaught AssertionError: expected 3072 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\cluster.js:144:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  8) Reconnect functionality should emit a disconnect and a reconnecting event after proper delay:

      Uncaught AssertionError: expected 1 to be 4
      + expected - actual

      -1
      +4

      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\cluster.js:149:26)
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:419:14)
      at TCP._handle.close [as _onclose] (net.js:498:12)

  9) Reconnect functionality should emit multiple reconnecting events and fail after maxReconnectAttempts:
     Uncaught AssertionError: expected 1023 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\reconnect.js:71:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  10) Reconnect functionality should emit reconnecting events indefinitely if maxReconnectAttempts is set to -1:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:22222
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  11) Reconnect functionality should succesfully reconnect to new server:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:22223
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  12) Reconnect functionality should succesfully reconnect to new server with subscriptions:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:22223
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  13) Reconnect functionality should succesfully reconnect to new server with queue subscriptions correctly:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  14) Reconnect functionality should properly resync with inbound buffer non-nil:
     Uncaught AssertionError: expected 2043 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\reconnect.js:71:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  15) Reconnect functionality should not crash when sending a publish with a callback after connection loss:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  16) Reconnect functionality should execute callbacks if published during reconnect:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  17) Reconnect functionality should not lose messages if published during reconnect:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  18) Reconnect functionality should emit reconnect before flush callbacks are called:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  19) Split Messages "before all" hook:
     Uncaught NatsError: Could not connect to server: Error: connect ECONNREFUSED 127.0.0.1:1426
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  20) Subscription Events "before all" hook:
     Uncaught AssertionError: expected 3071 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\reconnect.js:71:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  21) Timeout and max received events for subscriptions "before all" hook:
     Uncaught AssertionError: expected 3215 to be within 20..100
      at Assertion.fail (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:231:17)
      at Assertion.value [as within] (E:\Repositorys\node-nats\node_modules\should\cjs\should.js:305:19)
      at Client.<anonymous> (E:\Repositorys\node-nats\test\reconnect.js:71:25)
      at Client.reconnect (E:\Repositorys\node-nats\lib\nats.js:1204:10)
      at Timeout._onTimeout (E:\Repositorys\node-nats\lib\nats.js:1230:34)

  22) TLS "before all" hook:
     Uncaught NatsError: Could not connect to server: Error: read ECONNRESET
      at Socket.<anonymous> (E:\Repositorys\node-nats\lib\nats.js:448:28)
      at emitErrorNT (net.js:1278:8)
      at _combinedTickCallback (internal/process/next_tick.js:74:11)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  23) TLS "after all" hook:
     TypeError: Cannot read property 'kill' of undefined
      at Context.<anonymous> (E:\Repositorys\node-nats\test\tls.js:40:14)
      at callFn (C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:334:21)
      at Hook.Runnable.run (C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:327:7)
      at next (C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runner.js:299:10)
      at Immediate.<anonymous> (C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runner.js:327:5)

  24) Double SUBS "before all" hook:
     Error: Can't connect to server on port: 1922
      at Timeout.<anonymous> (E:\Repositorys\node-nats\test\support\nats_server_control.js:58:14)

  25) Yield "before all" hook:
     Uncaught TypeError: Cannot set property 'state' of undefined
      at C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runner.js:562:20
      at done (C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:291:5)
      at C:\Users\dustin\AppData\Roaming\npm\node_modules\mocha\lib\runnable.js:372:7
      at finish (E:\Repositorys\node-nats\test\support\nats_server_control.js:48:7)
      at Socket.<anonymous> (E:\Repositorys\node-nats\test\support\nats_server_control.js:70:9)
      at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1074:10)

[aricart]

@StarpTech Thanks for the report. @ColinSullivan1 is going to help me out with this on windows as I don't have a windows setup. Thanks for the heads up.

[StarpTech]

Anyone working on it?

[ColinSullivan1]

@StarpTech , yes, we are.

[StarpTech]

@ColinSullivan1 cool if you need any help dont hesitate to ask me. After this I can start with issue #108

[ColinSullivan1]

@StarpTech Thanks, appreciate the offer, and we may take you up on that. My initial take here is there are two problems - first, mkdir -p reports/ is not friendly to windows - we may just suggest different instructions for running the tests on windows (likely mocha as you have done). The second issue is the failing tests. I'm digging into those now.

[StarpTech]

@ColinSullivan1 you can use https://github.com/substack/node-mkdirp it common to use this for cross platform support.

[StarpTech]

E.g

 "test:unit": "./node_modules/mkdirp reports/" ...

[ColinSullivan1]

@StarpTech Much appreciated! We'll give that a shot.

Please don't let this block #108 - For testing your PR for #108, feel free to go ahead and write a test, and so long as that individual test passes for you under mocha, you should be good to go. The basic tests are passing in your environment, and Travis CI will catch any bugs that would be masked by the test failures here. Beyond the problem with mkdir, I'm fairly confident the test failures you are seeing here stem from the longer time it takes to establish TCP connections on Windows.

Thanks,
Colin

[ColinSullivan1]

@StarpTech , Just giving you an update - @aricart and I discussed this issue. We realize this is an issue, but believe that the problems lie in the tests themselves being run in a windows environment, rather than the node client itself. We have plans to address this, but in the meantime, are confident Travis CI will thoroughly test the node client. We'll leave this issue open until we come to a resolution.

[StarpTech]

@colinsullivan I would agree when it works on a Windows CI but with this decision you ignore windows users. I created a nats project on windows and my test suite passed. It needs just more investigation. Thanks.

[ColinSullivan1]

@StarpTech , I appreciate your concern! ...like I said, we do have plans to address this - we aren't ignoring windows users. It had been a number of days since the issue was updated and I was just reiterating that we do not feel this issue should block contributions until it's been resolved. This is certainly on our list of things to fix.

[StarpTech]

Hi @ColinSullivan1 sounds good but one advise I wouldn't use this driver when it breaks on my machine. I can estimate this issue but lots of people dont. I will have a look at this too, perhaps I can find the issues.

[StarpTech]

Any progress? 😱

[aricart]

Not yet - I'll see what I can do for v2.