Incorrect code line count caused by ChildLanguage

[LeoQ7]

const demo: &str = "Hello, World!";

/// markdown? docstring?

Output:

> tokei poc.rs
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            0            0            0            0
 |- Markdown             1            1            0            1            0
 (Total)                              1            0            1            0
===============================================================================
 Total                   1            0            0            0            0
===============================================================================

Trace:

[2024-03-06T18:24:01Z TRACE tokei::language::language_type] const demo: &str = "Hello, World!";
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Start "\""
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] End "\""
[2024-03-06T18:24:01Z TRACE tokei::language::syntax]

[2024-03-06T18:24:01Z TRACE tokei::language::syntax]

[2024-03-06T18:24:01Z TRACE tokei::language::syntax] /// markdown? docstring?
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?"
[2024-03-06T18:24:01Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-06T18:24:01Z TRACE tokei::language::syntax] Comment No.1

Expected output:

> tokei poc.rs
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            3            1            1            1
 (Total)                              3            1            1            1
===============================================================================
 Total                   1            3            1            1            1
===============================================================================

[XAMPPRocky]

Thank you for your issue! However tokei is correct here. All doc comments in Rust are markdown and this code is invalid Rust because it needs to be attached to an item.

[LeoQ7]

Thank you for the clarification!

For the following code snippet, is the result from tokei also correct?

const demo: &str = "Hello, World!";

/// markdown? docstring?
const demo1: &str = "Hello, World!";

Output:

[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] End "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax]

[2024-03-07T04:45:50Z TRACE tokei::language::syntax]

[2024-03-07T04:45:50Z TRACE tokei::language::syntax] /// markdown? docstring?

[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?\n"
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Comment No.1
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T04:45:50Z TRACE tokei::language::syntax] End "\""
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T04:45:50Z TRACE tokei::language::language_type] Code No.1
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            1            1            0            0
 |- Markdown             1            1            0            1            0
 (Total)                              2            1            1            0
===============================================================================
 Total                   1            1            1            0            0
===============================================================================

IMHO, LoC should be 2 here?

A correct example is:

const demo: u8 = 42;

/// markdown? docstring?
const demo1: &str = "Hello, World!";

Output:

[2024-03-07T06:18:40Z TRACE tokei::language::language_type] Using Simple Parse on "const demo: u8 = 42;\n\n"
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] /// markdown? docstring?
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] /// markdown? docstring?

[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Markdown found: " markdown? docstring?\n"
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] markdown? docstring?
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] ^ Skippable
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Comment No.1
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] Start "\""
[2024-03-07T06:18:40Z TRACE tokei::language::syntax] End "\""
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] const demo1: &str = "Hello, World!";
[2024-03-07T06:18:40Z TRACE tokei::language::language_type] Code No.1
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Rust                    1            3            2            0            1
 |- Markdown             1            1            0            1            0
 (Total)                              4            2            1            1
===============================================================================
 Total                   1            3            2            0            1
===============================================================================

[LeoQ7]

A similar issue is #713 , my guess is that the early return at https://github.com/XAMPPRocky/tokei/blob/master/src/language/syntax.rs#L250-L252 somehow affected the counting of previous lines when there is a quote.