Skip to content

NTVS 1.2 Manual Test Matrix

Jump to bottom
Matt Bierner edited this page Jul 27, 2016 · 12 revisions

Manual testing matrix and test scenarios for NTVS 1.2

Setup

  1. Windows versions:

  2. Windows 7

  3. Windows 8.1

  4. Windows 10

  5. Install Visual Studio, select custom settings, and make sure the Web Tools component is checked during the install.

  6. Visual studio 2015 + Update 2, Visual Studio 2015 + Update 3 (latest bits) 1. Community 2. Enterprise (and Pro) 3. Express for Web

  7. Additionally, please run a partial test on the following (single-configuration only, only walk through setup steps, not scenarios) 1. Visual Studio 2015 RTM 2. Visual Studio 2015 + Update 1

  8. Language settings

  9. English VS on English Windows

  10. Chinese VS on English Windows

  11. German VS on English Windows

  12. English VS on Chinese Windows

  13. Chinese VS on Chinese Windows

  14. English VS on German Windows

  15. German VS on English Windows

  16. Install Version of NTVS to upgrade from (to test upgrade paths):

  17. VS 2015 RTM: 1. No previously installed NTVS version

  18. VS 2015 + Update 1: 1. NTVS 1.1: http://aka.ms/ntvs.1.1.2015 (make sure to boot VS, create new NTVS project, and open interactive window with NTVS 1.1 installed before upgrading.)

  19. VS 2015 + Update 2, VS 2015 + Update 3 (latest bits): 1. No previously installed NTVS version 2. NTVS 1.1: http://aka.ms/ntvs.1.1.2015 (make sure to boot VS, create new NTVS project, and open interactive window with NTVS 1.1 installed before upgrading.)

  20. Install Version of NTVS to upgrade to (NTVS 1.2): https://github.com/Microsoft/nodejstools/releases/download/v1.2.Dev-6.13.2016/NTVS.Dev.2016-06-13.VS.2015.msi

  • ⭐Expected (VS 2015 + Update 3): Install should complete
  • ⭐Expected (VS 2015 RTM, VS 2015 + Update 1, VS 2015 + Update 2): Should receive an error message during install, and install should be aborted after clicking "Abort Install". Test pass for this configuration is complete - do not run through further scenarios.
  1. Install Node.js (test all versions)
  1. Install Azure Tools [optional, but required for web publishing support]
  2. Azure Tools for VS 2015: http://go.microsoft.com/fwlink/?LinkId=534216
  3. [OPTIONAL] Other extensions
  4. Web Essentials 1. VS 2015: https://visualstudiogallery.msdn.microsoft.com/ee6e6d8c-c837-41fb-886a-6b50ae2d06a2
  5. ReSharper: https://www.jetbrains.com/resharper/

Make sure everything installed OK

Scenarios

Create New Project

Checks creation of new NTVS projects from templates.

Prerequisites

None

Testing

  • Go to File -> New -> Project

  • Select the Templates -> JavaScript -> Node.js -> Starter Azure Node.js Express 3 template

  • Save the new project in c:\src

    • ⭐Expected: New project should open in Visual Studio without any error messages.

  • Wait for npm install to finish executing in the background (see details in the Output npm pane)

    • ⭐Expected: Results of npm install command are printed to output pane.
    • ⭐Expected: No error messages were printed to the output pane.
    • ⭐Expected: express, jade and stylus are listed under the npm node of the solution explorer.
Other Cases 💼

Also run through this entire scenario using a new Typescript project, from the template: Templates -> TypeScript -> Node.js -> Starter Azure Node.js Express 3

Basic Usage

Checks basic running and debugging functionality.

Prerequisites

Complete after Create New Project, and make sure to run this scenario for all cases 💼 of that scenario.

Testing
  • Set a breakpoint in app.js on the line (or app.ts):
var app = express()
  • Set a breakpoint in routes\index.js (or routes\index.ts) at the line that begins with:
res.render('index'

  • F5 to start debugging

    • ⭐Expected: Ensure the browser launches (http://localhost:1337)
    • ⭐Expected: The first breakpoint is app is hit.

  • Continue program execution.

    • ⭐Expected: The breakpoint in routes/index is hit.

  • Continue program execution.

    • ⭐Expected: In the browser, the page now loads successfully.

Basic Debugger Stepping

Checks basic debug stepping

Prerequisites

None

Testing
  • Templates -> JavaScript -> Node.js -> Blank Node.js Console Application
  • In app.js (or app.ts), enter the following text:
function count(remaining) {
    if (remaining <= 0)
        return 0;
    return remaining + count(remaining - 1);
}

var sum = 0;
sum = count(3);
sum = count(5);

  • Set a breakpoint in count on the line sum = count(3);

  • Start debugging the program.

    • ⭐Expected: Program runs to the line sum = count(3);

  • Hover over the sum variable in the source text.

    • ⭐Expected: A value of 3 is displayed.

  • Hit step over in the debugger:

    • ⭐Expected: the program continues to the line sum = count(5);
    • ⭐Expected: Hovering over sum now shows a value of 6`.

  • Now hit step into in the debugger.

    • ⭐Expected: The debugger is now on the first line of the count function.
    • ⭐Expected: The callstack window lists an entry for count at the top of the stack.
    • ⭐Expected: The value of remaining is 5.

  • Hit step over to get to the line return remaining + count(remaining - 1); and then hit step into again

    • ⭐Expected: The debugger is now on the first line of the count function again.
    • ⭐Expected: The callstack window list two entries for count on the top of the stack.
    • ⭐Expected: The value of remaining is now 4.

  • Hit step out in the debugger:

    • ⭐Expected: The debugger is now on the last line of the count function.
    • ⭐Expected: The callstack window lists one entry for count at the top of the stack.
    • ⭐Expected: The value of remaining is 5.

  • Hit step out once again:

    • ⭐Expected: The debugger is now on the line sum = count(5)
    • ⭐Expected: The callstack does not have any entries for count.

  • Hit step over once:

    • ⭐Expected: The debugger is now on the line after sum = count(5)
    • ⭐Expected: The value of sum is 15
Other Cases 💼

Also run through this entire scenario using a new Typescript project, from the template: Templates -> TypeScript -> Node.js -> Blank Node.js Console Application

Profiling

Checks basic profiling functionality.

Prerequisites

Complete after Create New Project, and make sure to run this scenario for all cases 💼 of that scenario.

Testing
  • Walk through the “Profiling” and “Start your profiling session” sections in the Profiling docs.
    • ⭐Expected: Verify behavior is as expected and results look like they do in all screenshots.

Npm Management using GUI

Checks that management of Node packages using the NPM GUI.

Prerequisites

Complete after Create New Project, and make sure to run this scenario for all cases 💼 of that scenario.

Testing
Other Cases 💼

Try switching to VS themes and running through the scenario again. Make sure things are readable and nothing looks out of place in the NPM window.

Test Explorer

Ensure that the test explorer works as expected.

Prerequisites

Complete after Create New Project, and make sure to run this scenario for all cases 💼 of that scenario.

Testing
  • Walk through the Test Explorer documentation
    • ⭐Expected: Verify behavior is as expected and results look like they do in all screenshots.

Basic Intellisense

Checks that basic IntelliSense functions as expected.

Prerequisites

None

Testing

  • File -> New Project -> JavaScript -> Node.js -> Blank Node.js Console Application

  • Go to Tools -> Options and then Text Editor -> Node.js -> IntelliSense.

    • ⭐Expected: Ensure you are in the ECMAScript 6 IntelliSense mode (listed at the top of the options page).

  • Create a new directory named hello.

  • Right click on the directory to add a new javascript item item1.js

    • ⭐Expected: hello/item1.js file should now exist and be open in editor.
    • ⭐Expected: In "ECMAScript 5"-mode, the top of the editor does not have a dropdown bar.
    • ⭐Expected: In "ECMAScript 6" mode, the top of the editor does have a dropdown bar dropdowns.

  • Type process. in the hello/item1.js file

    • ⭐Expected: After typing the ., you should see a list of completions.
    • ⭐Expected: arch appears in the completions list, with a (non-warning) glyph to the left-hand side of it.

  • Change hello/item1.js to:

var fs = require('fs');
fs.
  • ⭐Expected: List of completions should be displayed after typing the . in fs.
  • ⭐Expected: writeFile Exists in the completions list, with a (non-warning) glyph to the left-hand side of it.
  • Change hello/item1.js to:
module.exports.a = 3;
module.exports.b = "hi";
  • Back in app.js, add the following at the end of the file:
var item = require('./hello/item1');
item.
  • ⭐Expected: List of completions should be displayed after typing the . in item., starting with a and b.
  • ⭐Expected: completion list items a and b have a (non-warning) glyph to the left-hand side of them.
  • Now select the completion for b so that the text file looks like this:
var item = require('./hello/item1');
item.b
  • Hover over the b and hit F12.
  • ⭐Expected: Editor opens ``hello/item1file and highlights themodule.exports.b = ...` line
Other Cases 💼

Switch to ECMAScript 5 intellisense in Tools -> Options and then Text Editor -> Node.js -> IntelliSense and make sure Full IntelliSense is selected. Start the scenario from scratch, and try running through it again.

Interactive Window in Simple Project

Interactive window can evaluate expressions and run simple npm commands.

Prerequisites

Complete after Create New Project, and make sure to run this scenario for all cases 💼 of that scenario.

Testing

  • With an open project.

  • Open the Node.js Interactive Window using Ctrl+K, N

    • ⭐Expected: Node.js Interactive window should now be open with no error messages.

  • Run var a; a = [1, 2, 3].map(function(x) { return x * x; })

    • ⭐Expected: The result [ 1, 4, 9] is printed

  • Press the up arrow.

    • ⭐Expected: The previous command (var a; a = [1, 2, 3].map(function(x) { return x * x; })) should now be in the active buffer.

  • Clear the active buffer and enter a.

    • ⭐Expected: The result [ 1, 4, 9] is printed again.

  • Run .npm install azure --save-dev in the interactive window and wait for the command to complete

    • ⭐Expected: There are no error or warning messages.
    • ⭐Expected: The output is readable and doesn’t include any odd characters.
    • ⭐Expected: An entry for azure is now listed in the npm -> Dev node in Solution Explorer.

Interactive Window without any open Projects

Interactive window can evaluate expressions when no projects are loaded

Prerequisites

None

Testing

  • Without any open project (such as first boot of VS)

  • Open the Node.js Interactive Window using Ctrl+K, N

    • ⭐Expected: Node.js Interactive window should now be open with no error messages.

  • Run var a; a = [1, 2, 3].map(function(x) { return x * x; })

    • ⭐Expected: The result [ 1, 4, 9] is printed

Automatic Typings Acquisition for new Javascript Project

Prerequisites

Make sure ECMAScript 6 IntelliSense is enabled in Tools -> Options and then Text Editor -> Node.js -> IntelliSense

Testing

  • Go to File -> New -> New Project

  • Select the Templates -> JavaScript -> Node.js -> Basic Node.js Express 4 App template

  • Save the new project in c:\src

    • ⭐Expected: New project should open in Visual Studio without any error messages.

  • Wait for npm install to complete:

  • Open the app.js top level file.

  • At the end of the file, type path.

    • ⭐Expected: After typing the ., you should see a list of completions for path, including basename and delimiter
    • ⭐Expected: completion list items path, basename, and delimiter have (non-warning) glyphs to the left-hand side.

  • Now type app.

    • ⭐Expected: After typing the ., you should see a list of completions for the express app, including all and apply
    • ⭐Expected: completion list items all, apply have (non-warning) glyphs to the left-hand side of them.
Other Cases 💼

  • Turn off Automatic typings acquisition in Tools -> Options and then Text Editor -> Node.js -> IntelliSense with Automatically add IntelliSense typings folder to Node.js project. Try running through the same scenario. You should NOT see a yellow warning and a typings file should NOT be included in the project now.

  • Switch to ECMAScript 5 IntelliSense in Tools -> Options and then Text Editor -> Node.js -> IntelliSense and try running through the same scenario. You should NOT see a yellow warning and a typings file should NOT be included in the project now.

  • Try creating an Express typescript project instead of a Javascript project. Typings should not be downloaded.

  • Try Disabling the Show status bar after adding typings option and running the scenario again. A typings folder should be downloaded but no yellow status message should appear.

Typings are Automaticlly Acquired for newly installed packages.

Prerequisites

Make sure that Automatic Typings Acquisition for new Javascript Project works first.

Make sure ECMAScript 6 IntelliSense is enabled in Tools -> Options and then Text Editor -> Node.js -> IntelliSense

Testing

  • In current project.

  • Open the Node.js interactive Window: Tools -> Node.js Tools -> Node.js Interactive Window

  • Run .npm install underscore --save in the interactive window and wait for the command to complete.

    • ⭐Expected: underscore should now be listed in the npm node of the Solution Explorer.

  • Check that typings were also installed for underscore

    • ⭐Expected: In the typings directory, there is a folder called underscore containing a file called underscore.d.ts.
    • ⭐Expected: The underscore.d.ts file is also included in the project.

  • Open any Javascript file.

  • At the end of the file, type:

var _ = require('underscore');
_.
  • ⭐Expected: After typing the ., you should see a list of completions for underscore, including apply and first
  • ⭐Expected: Completion list items apply and first have (non-warning) glyphs to the left-hand side of them
Other Cases 💼

Typings acquisition should not happen the same case listed in Automatic Typings Acquisition for new Javascript Project.

Clone this wiki locally