[Obsolete draft] dev(testing/docker): Add support for local, Docker-based testing #234
Conversation
hughsw
force-pushed
the
docker-testing-sq1
branch
from
fff7e87
to
5a94ac3
Compare
| # Dev-centric test steps based on scripts in package.json | ||
| PATH=$(npm bin):$PATH | ||
|
|
||
| # If ava testing hangs (which, sadly, happens all too often), add --serial to get a more |
There was a problem hiding this comment.
I haven't been seeing ava hang - do you have any repro steps? What's your system configuration?
There was a problem hiding this comment.
Heh, this PR captures my system config. ;-) It's Docker for Mac. Within this framework it's intermittent, and less likely when the --serial option is given to ava. There are perhaps three tests that sometimes fail. But, given ava's relative secrecy about what's being run, it's hard to figure out which test is hung. I'm new to a lot of the tools you're using, so I'm a bit tentative about my conclusions.
If I run Node from the command-line on the Mac (using the brew-installed Node 10), the tests almost never succeed..., that's what got me working on Docker and Bee-Queue.
There was a problem hiding this comment.
Yeah ava was an early adoption and while on the one hand I'd kinda prefer jest for its stability and support I do still like the raw speed that ava provides. If only the author of ava-spec didn't deprecate it in favor of Jest...
I might have to try and get my hands on a mac to repro what you're seeing - it'd be nice if these tests succeeded, and race conditions like the ones you're talking about concern me.
There was a problem hiding this comment.
I will spend a little time on looking at this PR/branch in a Docker system on an underlying Linux kernel (unlike Mac, which uses the moby kernel emulator or whatever you call it).
There was a problem hiding this comment.
Experiment:
I used the PR branch on a Digital Ocean droplet which has an Ubuntu-based Docker environment -- So, stable, cloud-centric Linux kernel and mainstream Ubuntu/Debian Docker.
I used this script, npmtest.sh:
#!/bin/sh
exec npm testRun via:
(while ./docker-tests.sh npmtest.sh ; do echo && echo timestamp && date && echo ; done) > /tmp/beequeuetests.log 2>&1
On this Droplet a new "timestamp" line would appear in the log file every 22 seconds or so, until... either:
- the while loop stopped, meaning an explicit test failure
- the number of "timestamp" lines in the log file stopped increasing, meaning a hung test
Out of 75 execs of npm test I got:
- 67 successful runs
- 7 hung tests
- 1 failure
So, overall 89% success rate.
The one failure (which I have occasionally seen on Mac):
bee-queue_1 | 139 passed
bee-queue_1 | 1 failed
bee-queue_1 |
bee-queue_1 | helpers-test › finallyRejectsWithInitial waits for the returned Promise
bee-queue_1 |
bee-queue_1 | /bee-queue/test/helpers-test.js:85
bee-queue_1 |
bee-queue_1 | 84: );
bee-queue_1 | 85: t.true(measure() >= 10);
bee-queue_1 | 86: t.true(stub.calledOnce);
bee-queue_1 |
bee-queue_1 | Value is not `true`:
bee-queue_1 |
bee-queue_1 | false
bee-queue_1 |
bee-queue_1 | measure() >= 10
bee-queue_1 | => false
bee-queue_1 |
bee-queue_1 | measure()
bee-queue_1 | => 7.332423
bee-queue_1 | npm ERR! Test failed. See above for more details.
bee-queue_bee-queue_1 exited with code 1
The 7 hung runs had all completed 139 of the 140 tests in this branch. I'll have to work a little to figure out which test(s) were the one(s) that hung.
There was a problem hiding this comment.
I have goodish news. So far, using my uuid-based approach (#237) to fixing the Job.id race condition, I am not seeing any of the non-deterministic test failures and hangs.
| # deterministic record that will log up to the test prior to the one that hangs... But, as | ||
| # is often the case with race conditions, the instrumentation may eliminate the problem | ||
| # you're trying to track down... | ||
| ava --no-color --timeout 30000 --verbose |
There was a problem hiding this comment.
I wouldn't mind adding a --timeout to the default test command.
| ava --no-color --timeout 30000 --verbose | ||
| eslint . | ||
| prettier --check '**/*.(html|json|md|sublime-project|ts|yml)' | ||
| nyc ava |
There was a problem hiding this comment.
This will re-run ava - what do you think about combining this with the initial ava invocation?
There was a problem hiding this comment.
We should discuss the purpose of this or other scripts we include. As it stands, this script supports a dev-cycle (my dev cycle) whereby you get stuff working, you mollify the styling police, you move towards deployable coverage, etc. I've gone back and forth on just doing npm test, but it fails too often and the output isn't that helpful when you start on forensic work.
There was a problem hiding this comment.
I'm specifically wondering about the inclusion of both ava and nyc ava in this script - not the use of a separate tests.sh file. Not sure if that was apparent from my original comment.
| # Ensure that tools for npm are available (for node-gyp in particular) | ||
| # to build node_modules |
There was a problem hiding this comment.
Are there any dependencies of bee-queue that actually require this?
There was a problem hiding this comment.
node-gyp is the bad boy here. Has been a PITA for years. Something in the set of bee-queue (dev?) packages needs it. We could get more nuanced if necessary, but this is a simple solution right now.
There was a problem hiding this comment.
Strange - that isn't happening for me. It's happening for you on alpine or just on mac?
There was a problem hiding this comment.
I was a bit terse yesterday, long day. It's somewhat complicated.
I do almost all my work using Docker for Mac on my MacBook. The Alpine in question is the base Docker image used in the FROM <base-image> statements in the Dockerfile. That is, it's a Node image built on Alpine, so giving about as small an image as possible. https://hub.docker.com/_/node/ the node:lts-alpine tag.
The other environment I occasionally use is an Ubuntu/Debian Docker on a Digital Ocean droplet.
nody-gyp is used by npm when C/C++ libraries need to be compiled for npm modules. I've seen it used for packages that need to wrangle bits and bytes to construct and/or unpack binary C structs to send over the wire, e.g. Sequelize to talk to SQLservers. I'm guessing that the redis library wants to do the same thing to talk to Redis -- just guessing.
AFAICT, npm has several options about how to proceed in a given npm install environment:
- pull in pre-compiled libraries for the platform
- pull in node-gyp tools and compile the C/C++ code
- fall back to a Javascript implementation
Using Alpine base image brings lots of these issues to the fore. There are far fewer pre-compiled packages, and no build system (or even bash). It can be a bit tedious, but it does help shed light on some subtle background matters. Using Debian or Ubuntu won't have this friction...
If you look at the Hello World example PR you'll see that I don't pull in the build system, and the example works. If you remove the --quiet option you'll see that there are numerous errors emitted by npm regarding not finding Python and that its "not OK". Yet, the install succeeds and the example works.
If I add apk add Python, the build then emits errors about make not being present, yet the example works. And if I add make then it emits errors about not having g++, etc.
I'm guessing that in the face of these errors, the build for redis falls back to a native JS implementation -- again, just guessing.
For the Hello World example this underlying turmoil is irrelevant, and whatever npm has installed is good enough.
But, for starting on a Docker configuration that wants to be reliable vis-a-vis Bee-Queue testing and performance, I want to use a robust build chain. So, that's what's in this PR. In the long run (if there is one) we would certainly want to have the testing vet both versions of the redis runtime (as Travis does with various Node versions). I would also want to build/test with a Debian base image for similar exposure to different underlying tool chains.
There was a problem hiding this comment.
And, I'm stubborn about continuing to use a frictional build chain if it helps us observe, isolate and correct race conditions that are not manifest in the standard CI chain.
There was a problem hiding this comment.
I'm guessing that the redis library wants to do the same thing to talk to Redis -- just guessing.
That was once the case - redis used a library called hiredis - but that hasn't been the case for a long time.
I think a general (minor) goal of this project is to not have any C/C++ dependencies, but I don't know that that needs to block this PR.
Co-Authored-By: Eli Skeggs <skeggse@users.noreply.github.com>
|
This PR is made obsolete by #239 |
hughsw
changed the title
[ This PR is made obsolete by #239 ]
See Testing section at bottom of README: https://github.com/hughsw/bee-queue/blob/docker-testing-sq1/README.md#testing