Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The next round of test work #3486

Open
wants to merge 11 commits into
base: dev
Choose a base branch
from
Open

The next round of test work #3486

wants to merge 11 commits into from

Conversation

nwf
Copy link
Member

@nwf nwf commented Dec 30, 2021

  • This PR is for the dev branch rather than for the release branch.
  • This PR is compliant with the other contributing guidelines as well (if not, please describe why).
  • I have thoroughly tested my contribution.

Here's a slice through the test framework I've had in my back pocket. Of importance:

  • it introduces, and demonstrates initial minimal use of, a NTestEnv module (and associated file in SPIFFS); this will let us flexibly extend the notion of "test environment" without requiring that everything move in lockstep.
  • the MQTT test program demonstrates the real power of the "harness" approach, IMHO
  • the top-level runner (tests/all.sh) shows how the whole thing hangs together

Thoughts more than welcome.

@nwf nwf added this to the 2022 rel 1 milestone Jan 2, 2022
Copy link
Member

@HHHartmann HHHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good approach. Happy to see things going on.
For extra points you could add a test for NTestEnv itself

Maybe it would make sense to allow for several parameters in hasLua and hasC.


-- Determine if a given Lua module is available for require-ing.
function NTE.hasL(m)
for _, l in ipairs(package.loaders) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use node.LFS.get()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see the difference. Your method will also find modules in SPIFFS.
But will it not load all available modules, that way exhausting memory if SPIFFS is not cleaned out?
But I am no expert in loaders.

-- Determine if the firmware has been built with a given module
function NTE.hasC(m)
-- this isn't great, but it's what we've got
local mstr = node.info('build_config').modules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could also just check if the name is available in the root namespace (and is a rom table maybe)

end

-- Determine if a given Lua module is available for require-ing.
function NTE.hasL(m)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer hasLua. On a host system (with more memory) I would even call it hasLuaModule


-- Look up one (or more) feature keys in our configuration file,
-- returning the associated value (or a map from keys to values).
function NTE.getFeat(...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about getFeatures. The plural indicates that several params can be given at once


local cstr
repeat
cstr = cfgf:read(); decoder:write(cstr or "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to have smaller chunks than the default 1024 bytes?
Should we call an explicit garbage collect from time to time?

assert (type(givenFeats) == "table", "Malformed configuration file")

local res = {}
for _, v in ipairs(reqFeats) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be done only on request.
I imagine a test that behaves differently on existence of a feature.
Maybe have on of

  • a separate function that errors on missing features
  • a parameter that steers the behavior
  • a separate function that is called with the result of getFeat and the list of required features to error out

if #p > 1 then return true end -- something we're keeping
if #p == 0 then return true end -- root table
local thisFeat = p[1]
assert (type(thisFeat) == "string")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about json syntax but are there other keys than strings allowed?
w3schools states: "In JSON, keys must be strings, written with double quotes"
so this check can be removed

-- Ensure that we're on DUT 0
assert(NTE.getFeat('DUT') == 0, "Not on DUT 0")

-- Configure the ADC (this implicitly checks for having the ADC module)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early checks with distinct error message are a nice thing to have. So since we now have respective methods we should use them. Maybe even have such checks for basic modules as sjson before using them in NTestEnv

if adc.force_init_mode(adc.INIT_ADC)
then
node.restart()
error "Must reboot to get to ADC mode"
end

-- Configure the I2C bus
-- Configure the I2C bus (again, implicitly testing...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again ... ;-)

Copy link
Member

@HHHartmann HHHartmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forgot to check where the deleted content was moved to

#### DHTxx

On DUT1 pin D6/GPIO 12 there is a connection to a position for a DHTxx device. The silk screen indicates the
orientation of the device.

Note: Not yet used in the test harness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean the software test harness? This is the documentation of the HWTH

Note: Not yet used in the test harness.

Note: It would make sense to augment the 1-Wire testing facility to
include bus-drive power, perhaps via the MCP23017, especially if we ever
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As states above there are already 2 devices possible, one without VCC connected which then would be bus-driven

#### DS18B20

There are two positions for DS18B20s -- one with the VCC pin connected and one without. The data pin is
connected to DUT1 pin D5/GPIO 14.

Note: Not yet used in the test harness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meaning no tests available yet?

---------|--------------------------------------------------------------
/RESET |DUT0 reset. This resets the chip whenever the host computer resets DUT 0 over its serial link (using DTR/RTS).
B 0 |4K7 resistor to DUT 0 ADC.
B 1 |2K2 resistor to DUT 0 ADC.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again B2,3 and 4 are not described in the readme



ESP8266 Device 0 Connections
----------------------------
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description here is more exact and exceeds the spec in the readme, so it should be kept.
e.g.; D4/5 of DUT 1 is not described at all.
Also in general the setup might evolve in the readme but this document describes a hardware version, which will remain unchanged (else it will be a v2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants