PowerShell 7.1 preview 3 doesn't use configured date/time format

[sba923]

This should speak for itself:

> $PSVersionTable.PSVersion.ToString(); (Get-ItemProperty 'HKCU:\Control Panel\International' -name sTimeFormat).sTimeFormat;  (Get-ItemProperty 'HKCU:\Control Panel\International' -name sShortDate).sShortDate; Get-Date -DisplayHint Time; Get-Date | ConvertTo-Csv
7.0.1
HH:mm:ss
yyyy-MM-dd

10:03:10
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 10:03:10","2020-05-22 00:00:00","22","Friday","143","10","Local","179","3","5","10","637257385901795718","10:03:10.1795718","2020"

> $PSVersionTable.PSVersion.ToString(); (Get-ItemProperty 'HKCU:\Control Panel\International' -name sTimeFormat).sTimeFormat;  (Get-ItemProperty 'HKCU:\Control Panel\International' -name sShortDate).sShortDate; Get-Date -DisplayHint Time; Get-Date | ConvertTo-Csv
7.1.0-preview.3
HH:mm:ss
yyyy-MM-dd

10:03:37 AM
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 10:03:37 AM","5/22/2020 12:00:00 AM","22","Friday","143","10","Local","61","3","5","37","637257386170614178","10:03:37.0614178","2020"

Steps to reproduce

• use the standard en-US culture
• set the time format to 24-hr HH:mm:ss and date format to yyyy-MM-dd via Settings
• run Get-Date | ConvertTo-Csv

Expected behavior

Time is formatted using 24-hr format, date using specified yyyy-MM-dd format.

Actual behavior

Time is formatted in 12-hr am/pm format, date using M/dd/yyyy format

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.1.0-preview.3
PSEdition                      Core
GitCommitId                    7.1.0-preview.3
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[vexx32]

Is this behaviour new in the latest preview or has it always been this way?

[sba923]

I would say it's new in preview 3. Too bad, I've just upgraded my last system, so I don't have any preview 2 system anymore.

[vexx32]

We'll need to confirm if there was a change at some point or if this is the original behaviour.

Also, can you confirm if this is affected at all by using the -UseCulture parameter explicitly?

[iSazonov]

We use Datetime.ToLongTimeString() method for output in the case.
Also you could look (Get-Culture).DateTimeFormat.

[sba923]

In 7.0.1:

> $PSVersionTable.PSVersion.ToString(); (Get-Culture).DateTimeFormat
7.0.1

AMDesignator                     : AM
Calendar                         : System.Globalization.GregorianCalendar
DateSeparator                    : -
FirstDayOfWeek                   : Monday
CalendarWeekRule                 : FirstDay
FullDateTimePattern              : dddd, MMMM d, yyyy HH:mm:ss
LongDatePattern                  : dddd, MMMM d, yyyy
LongTimePattern                  : HH:mm:ss
MonthDayPattern                  : MMMM d
PMDesignator                     : PM
RFC1123Pattern                   : ddd, dd MMM yyyy HH':'mm':'ss 'GMT'
ShortDatePattern                 : yyyy-MM-dd
ShortTimePattern                 : HH:mm
SortableDateTimePattern          : yyyy'-'MM'-'dd'T'HH':'mm':'ss
TimeSeparator                    : :
UniversalSortableDateTimePattern : yyyy'-'MM'-'dd HH':'mm':'ss'Z'
YearMonthPattern                 : MMMM yyyy
AbbreviatedDayNames              : {Sun, Mon, Tue, Wed…}
ShortestDayNames                 : {Su, Mo, Tu, We…}
DayNames                         : {Sunday, Monday, Tuesday, Wednesday…}
AbbreviatedMonthNames            : {Jan, Feb, Mar, Apr…}
MonthNames                       : {January, February, March, April…}
IsReadOnly                       : True
NativeCalendarName               : Gregorian Calendar
AbbreviatedMonthGenitiveNames    : {Jan, Feb, Mar, Apr…}
MonthGenitiveNames               : {January, February, March, April…}

In 7.1 preview 3:

> $PSVersionTable.PSVersion.ToString(); (Get-Culture).DateTimeFormat
7.1.0-preview.3

AMDesignator                     : AM
Calendar                         : System.Globalization.GregorianCalendar
DateSeparator                    : /
FirstDayOfWeek                   : Sunday
CalendarWeekRule                 : FirstDay
FullDateTimePattern              : dddd, MMMM d, yyyy h:mm:ss tt
LongDatePattern                  : dddd, MMMM d, yyyy
LongTimePattern                  : h:mm:ss tt
MonthDayPattern                  : MMMM d
PMDesignator                     : PM
RFC1123Pattern                   : ddd, dd MMM yyyy HH':'mm':'ss 'GMT'
ShortDatePattern                 : M/d/yyyy
ShortTimePattern                 : h:mm tt
SortableDateTimePattern          : yyyy'-'MM'-'dd'T'HH':'mm':'ss
TimeSeparator                    : :
UniversalSortableDateTimePattern : yyyy'-'MM'-'dd HH':'mm':'ss'Z'
YearMonthPattern                 : MMMM yyyy
AbbreviatedDayNames              : {Sun, Mon, Tue, Wed…}
ShortestDayNames                 : {S, M, T, W…}
DayNames                         : {Sunday, Monday, Tuesday, Wednesday…}
AbbreviatedMonthNames            : {Jan, Feb, Mar, Apr…}
MonthNames                       : {January, February, March, April…}
IsReadOnly                       : True
NativeCalendarName               : Gregorian Calendar
AbbreviatedMonthGenitiveNames    : {Jan, Feb, Mar, Apr…}
MonthGenitiveNames               : {January, February, March, April…}

[sba923]

We'll need to confirm if there was a change at some point or if this is the original behaviour.

Also, can you confirm if this is affected at all by using the -UseCulture parameter explicitly?

> $PSVersionTable.PSVersion.ToString(); Get-Date | ConvertTo-Csv -UseCulture
7.0.1
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 19:33:59","2020-05-22 00:00:00","22","Friday","143","19","Local","174","33","5","59","637257728391742480","19:33:59.1742480","2020"

> $PSVersionTable.PSVersion.ToString(); Get-Date | ConvertTo-Csv -UseCulture
7.1.0-preview.3
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 7:34:45 PM","5/22/2020 12:00:00 AM","22","Friday","143","19","Local","63","34","5","45","637257728850632108","19:34:45.0632108","2020"

[vexx32]

Looks like a regression indeed. @iSazonov do you know if there were any changes merged to this area of code since 7.1-p2? I can't recall any at the moment.

Perhaps it's a regression or change in .NET Core?

[sba923]

Here's what I get with preview 2 on the same system:

> $PSVersionTable.PSVersion.ToString(); (Get-ItemProperty 'HKCU:\Control Panel\International' -name sTimeFormat).sTimeFormat;  (Get-ItemProperty 'HKCU:\Control Panel\International' -name sShortDate).sShortDate; Get-Date -DisplayHint Time; Get-Date | ConvertTo-Csv
7.1.0-preview.2
HH:mm:ss
yyyy-MM-dd

19:42:05
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 19:42:05","2020-05-22 00:00:00","22","Friday","143","19","Local","14","42","5","5","637257733250149958","19:42:05.0149958","2020"

> $PSVersionTable.PSVersion.ToString(); Get-Date | ConvertTo-Csv -UseCulture
7.1.0-preview.2
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Friday, May 22, 2020 19:42:40","2020-05-22 00:00:00","22","Friday","143","19","Local","847","42","5","40","637257733608470239","19:42:40.8470239","2020"

[iSazonov]

I can not confirm. After I changed these format strings in registry and restarted PowerShell I get right outputs.

[vexx32]

What? Where was changing the format strings in the registry mentioned here? 😕

[sba923]

To get things right I uninstalled preview 3 and installed preview 2

😜

[ghost]

This issue has been marked as answered and has not had any activity for 1 day. It has been closed for housekeeping purposes.

[sba923]

I don't understand: there is an issue in preview 3, it has to be fixed!

[iSazonov]

I can repo. We need reliable repo steps.
You could download zip packages, install PowerShell side-by-side, run both version and check result (better on clean system).

[sba923]

You mean you can't repro?

I don't have a clean system to test on (could set up a test VM, but would take a significant amount of time).

I'll test with XCOPY-deployed SxS installations of preview 2 and preview 3.

[sba923]

Here's the outcome, using these launch scripts:

REM runpwsh.bat
cls
PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
.\pwsh -noprofile -file .\runtest.ps1

# runtest.ps1
$PSVersionTable.PSVersion.ToString()
(Get-ItemProperty 'HKCU:\Control Panel\International' -name sTimeFormat).sTimeFormat
(Get-ItemProperty 'HKCU:\Control Panel\International' -name sShortDate).sShortDate
Get-Date -DisplayHint Time
Get-Date | ConvertTo-Csv

Repro'ed on two different systems.

[iSazonov]

What is output:

$Host

(Get-Culture).DateTimeFormat

[sba923]

On preview 2:

C:\tmp\PowerShell-7.1.0-preview.2-win-x64>PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem

C:\tmp\PowerShell-7.1.0-preview.2-win-x64>.\pwsh -noprofile -file .\runtest.ps1
7.1.0-preview.2

Name             : ConsoleHost
Version          : 7.1.0-preview.2
InstanceId       : 91f11a8e-5486-4058-b1e8-9bb93bb9adc9
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : en-US
CurrentUICulture : en-US
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

AMDesignator                     : AM
Calendar                         : System.Globalization.GregorianCalendar
DateSeparator                    : -
FirstDayOfWeek                   : Monday
CalendarWeekRule                 : FirstDay
FullDateTimePattern              : dddd, MMMM d, yyyy HH:mm:ss
LongDatePattern                  : dddd, MMMM d, yyyy
LongTimePattern                  : HH:mm:ss
MonthDayPattern                  : MMMM d
PMDesignator                     : PM
RFC1123Pattern                   : ddd, dd MMM yyyy HH':'mm':'ss 'GMT'
ShortDatePattern                 : yyyy-MM-dd
ShortTimePattern                 : HH:mm
SortableDateTimePattern          : yyyy'-'MM'-'dd'T'HH':'mm':'ss
TimeSeparator                    : :
UniversalSortableDateTimePattern : yyyy'-'MM'-'dd HH':'mm':'ss'Z'
YearMonthPattern                 : MMMM yyyy
AbbreviatedDayNames              : {Sun, Mon, Tue, Wed…}
ShortestDayNames                 : {Su, Mo, Tu, We…}
DayNames                         : {Sunday, Monday, Tuesday, Wednesday…}
AbbreviatedMonthNames            : {Jan, Feb, Mar, Apr…}
MonthNames                       : {January, February, March, April…}
IsReadOnly                       : True
NativeCalendarName               : Gregorian Calendar
AbbreviatedMonthGenitiveNames    : {Jan, Feb, Mar, Apr…}
MonthGenitiveNames               : {January, February, March, April…}

On preview 3:

C:\tmp\PowerShell-7.1.0-preview.3-win-x64>PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem

C:\tmp\PowerShell-7.1.0-preview.3-win-x64>.\pwsh -noprofile -file .\runtest.ps1
7.1.0-preview.3

Name             : ConsoleHost
Version          : 7.1.0-preview.3
InstanceId       : 9244eab6-c591-4b06-9148-719bc5cba22d
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : en-US
CurrentUICulture : en-US
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

AMDesignator                     : AM
Calendar                         : System.Globalization.GregorianCalendar
DateSeparator                    : /
FirstDayOfWeek                   : Sunday
CalendarWeekRule                 : FirstDay
FullDateTimePattern              : dddd, MMMM d, yyyy h:mm:ss tt
LongDatePattern                  : dddd, MMMM d, yyyy
LongTimePattern                  : h:mm:ss tt
MonthDayPattern                  : MMMM d
PMDesignator                     : PM
RFC1123Pattern                   : ddd, dd MMM yyyy HH':'mm':'ss 'GMT'
ShortDatePattern                 : M/d/yyyy
ShortTimePattern                 : h:mm tt
SortableDateTimePattern          : yyyy'-'MM'-'dd'T'HH':'mm':'ss
TimeSeparator                    : :
UniversalSortableDateTimePattern : yyyy'-'MM'-'dd HH':'mm':'ss'Z'
YearMonthPattern                 : MMMM yyyy
AbbreviatedDayNames              : {Sun, Mon, Tue, Wed…}
ShortestDayNames                 : {S, M, T, W…}
DayNames                         : {Sunday, Monday, Tuesday, Wednesday…}
AbbreviatedMonthNames            : {Jan, Feb, Mar, Apr…}
MonthNames                       : {January, February, March, April…}
IsReadOnly                       : True
NativeCalendarName               : Gregorian Calendar
AbbreviatedMonthGenitiveNames    : {Jan, Feb, Mar, Apr…}
MonthGenitiveNames               : {January, February, March, April…}

[iSazonov]

What is time you change the datetime format?

The PowerShell versions use different .Net 5 Preview - 3 and 4.
You could download nightly build (it is on .Net 5.0 Preview5) and check again.
If you will see the issue you could discuss this in .Net Runtime repo.

[sba923]

I'm not sure I understand your question. When did I change the datetime format? That must've been when I installed the systems in the first place, 'cos as a software engineer I prefer 1) the English UI, so I changed the display language to English (my computers came with French Windows) 2) the ISO8601 YYYY-MM-DD date format. So I've probably never run any version of PowerShell with the default en-US date/time formats.

Where can I get the nightly builds from?

What is the .NET Core API that's used by PowerShell Core to convert a date/time to string? I need this information if I want to open an issue on .NET 5.0.

[iSazonov]

You can download night build from main page of the repository.

PowerShell uses Datetime.ToLongTimeString() method. All formats you get above with (Get-Culture).DateTimeFormat.

[sba923]

On that main page, I can see the status of the nightly builds in CI, but no download links. Am I missing something?

I'll try creating a C# test program and run it with various previous versions of .NET 5.

[iSazonov]

Press on the status icon and look artifacts under "Build for Windows".

[sba923]

Same repro:

[vexx32]

The code that handles this is here:

PowerShell/src/Microsoft.PowerShell.Commands.Utility/commands/utility/CsvCommands.cs

Lines 1045 to 1067 in b1e9980

	internal static string GetToStringValueForProperty(PSPropertyInfo property)
	{
	if (property == null)
	{
	throw new ArgumentNullException(nameof(property));
	}
	string value = null;
	try
	{
	object temp = property.Value;
	if (temp != null)
	{
	value = temp.ToString();
	}
	}
	catch (Exception)
	{
	// If we cannot read some value, treat it as null.
	}
	return value;
	}

With this code in mind, can you check what you get back for (Get-Date).ToString() and (Get-Date).PSObject.ToString()?

I'm reopening this as it doesn't look like we've gotten to the bottom of it yet. 🙂

[iSazonov]

It seems .Net has an option to opt out ICU. You could try and see a result.

[sba923]

Here's what I get with "System.Globalization.Invariant": true if this is what you were referring to:

Formatting is incorrect whatever the .NET 5 preview version.

[iSazonov]

No, there is an option to switch back from ICU to Windows traditional API.

[iSazonov]

See https://docs.microsoft.com/en-us/dotnet/core/run-time-config/globalization#nls

Globalization config settings - .NET Core

Learn about run-time settings that configure globalization aspects of a .NET Core app, for example, how it parses Japanese dates.

[sba923]

Will test... The change to use ICU was introduced in .NET 5 preview 4...

[sba923]

Bingo!

The .NET folks marked my issue as a duplicate of dotnet/runtime#37121, where @safern claims this was fixed for .NET 5 Preview 5--not what I see...

Am I correct to assume that you will not want to use NLS in PowerShell? IMVHO this would be a potential cause for PowerShell behaving differently on Windows and on other platforms.

[safern]

The .NET folks marked my issue as a duplicate of dotnet/runtime#37121, where @safern claims this was fixed for .NET 5 Preview 5--not what I see...

That claim from my side was initially wrong as I thought dotnet/runtime#37121 was a dupe of dotnet/runtime#35638 which was fixed in Preview5. I expect to have a fix for dotnet/runtime#37121 this week and hopefully will be able to port it to Preview6 on time.

[iSazonov]

Am I correct to assume that you will not want to use NLS in PowerShell?

PowerShell follows .Net (and OS) improvements. It would be amazing for users to discover that PowerShell works differently than other .Net and native OS applications.

[sba923]

@iSazonov I'm not sure I know how to interpret your answer. Are you (on behalf of the PowerShell team) in favor of aligning PowerShell on Windows with all other Windows applications (in which case NLS is the norm), or of aligning PowerShell on Windows with PowerShell on other platforms (in which case ICU's behavior will rule)?

[sba923]

For now, I might go for setting DOTNET_SYSTEM_GLOBALIZATION_USENLS=true in my global environment so that I get the expected behavior when running PowerShell 7.1 preview 3. Or I'll stick to using PowerShell 7.0.2 for any affected workloads.

[iSazonov]

@sba923 Since NLS and ICU have differences it can be a problem - one .Net application can work differently on different OS-es. MSFT has started to migrate to ICU. It was added in latest Window versions and .Net 5.0 uses ICU by default.
PowerShell 7.1 gets the enhancement automatically. The same is true for all .Net 5+ applications.

[sba923]

So you'll go for PS7.1 using ICU, and assuming this is backwards compatible e.g. abides by the user's settings, correct?

[iSazonov]

@sba923 7.1 will be on .Net 5.0. If the .Net will use ICU by default PowerShell will do the same.

[sba923]

Should be OK, provided this does not introduce severe breaking changes.

[sba923]

Expectedly, the issue is still present in 7.1 preview 4...

[safern]

To give an update I have a PR out for the fix: dotnet/runtime#38372

[sba923]

To give an update I have a PR out for the fix: dotnet/runtime#38372

Good! See my comment there about taking user changes into account on the fly. Does this need a code change on the PowerShell side of things?

[iSazonov]

If PowerShell 7.0 version sees the time (culture) changes immediately, then we get a breaking change.

[sba923]

If PowerShell 7.0 version sees the time (culture) changes immediately, then we get a breaking change.

How do you mean? That 7.1 should behave the same as 7.0? I don't know what 7.0's behavior is, but I would expect that it takes user changes into account dynamically.

[Liturgist]

PS C:\src\t> ($PSVersionTable.PSVersion).ToString(),(Get-Culture).DateTimeFormat.ShortDatePattern
7.1.0-preview.3
M/d/yyyy

PS C:\src\t> ($PSVersionTable.PSVersion).ToString(),(Get-Culture).DateTimeFormat.ShortDatePattern
7.0.1
yyyy-MM-dd

[iSazonov]

.Net fix was merged. We get it soon. (.Net 5.0 Preview8)

[sba923]

.Net fix was merged. We get it soon. (.Net 5.0 Preview8)

What's the plan for testing? How difficult is it to make a private build of PS7.1 with that .NET fix in so that we can test for side effects / regressions ?

[iSazonov]

@sba923 You could build .Net locally and test. But it is not worth the effort. We can wait about a month.

[sba923]

Still waiting for .NET Preview 8 to be released...

[rjmholt]

PowerShell 7.1.0-preview.7 ships with .NET Preview 8, which should address this issue. Let us know if it hasn't.

[sba923]

I concur 😜

> $PSVersionTable.PSVersion.ToString(); (Get-ItemProperty 'HKCU:\Control Panel\International' -name sTimeFormat).sTimeFormat;  (Get-ItemProperty 'HKCU:\Control Panel\International' -name sShortDate).sShortDate; Get-Date -DisplayHint Time; Get-Date | ConvertTo-Csv
7.1.0-preview.7
HH:mm:ss
yyyy-MM-dd

18:08:34
"DisplayHint","DateTime","Date","Day","DayOfWeek","DayOfYear","Hour","Kind","Millisecond","Minute","Month","Second","Ticks","TimeOfDay","Year"
"DateTime","Thursday, September 10, 2020 18:08:35","2020-09-10 00:00:00","10","Thursday","254","18","Local","25","8","9","35","637353581150253074","18:08:35.0253074","2020"