Attributes don't show up in array of hashtables

[gabrielsroka]

I tried the script Dr. Scripto wrote about in 2013:
https://devblogs.microsoft.com/scripting/write-users-and-proxy-addresses-to-csv-by-using-powershell/

but it didn't work. Perhaps it's no surprise since it was written many years ago.

It seems that PowerShell doesn't like this:

$users = @()

$user1 = [ordered]@{first = 1; last = 2}
$user2 = [ordered]@{first = 3; last = 5; middle = 5}

$users += [pscustomobject]$user1
$users += [pscustomobject]$user2

# The `middle` attribute doesn't show up in any of these:
$users
$users | Out-GridView
$users | ConvertTo-Csv -NoTypeInformation 
$users | Export-Csv users.csv -NoTypeInformation

But if you add $user2 first, it works fine.

I have a script that calls an API hundreds of times, and will get different user attributes each time.

I can find a workaround, but doesn't this seem like a bug?

Thanks!

[iSazonov]

It is side effect from Format-Table optimization when it is trying to collect some object before output.
For the case this looks like a bug.

[vexx32]

No, this is by design for this cmdlet. Isn't it in the documentation that the properties on the first object are used to determine the table headers?

Think about it. CSV requires that all lines have the same number of columns and that the column headings be listed at the top of the file.

Short of caching all the objects in memory and iterating the entire collection at least twice before writing the file, there's no way to ensure you haven't missed any properties and don't have a mismatched number of columns on some lines.

It would be extremely bad on performance for large CSV files to try to handle this in any other way imo.

If you have a set of objects that you know the full set of properties for, you can workaround this by using select-object to ensure that all the objects have the same set of properties before exporting them.

$objects |
    Select-Object -Property $PropertyNames |
    Export-Csv -Path $filepath

[gabrielsroka]

@vexx32 :

Isn't it in the documentation that the properties on the first object are used to determine the table headers?

Do you know where that is?

I tried this right after, and it does what I want.

$users | ft -Property first,last,middle 

but that's not intuitive. Plus, Export-Csv doesn't support it, so you need the Select workaround above (which is a good workaround, but can we "fix the bug" instead?)

Python, by comparison, requires you to name the columns you're exporting (like ft above)"

# Python:
fieldnames = ['first', 'last', 'middle']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

[gabrielsroka]

I found it, from https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/format-table?view=powershell-7

If you omit [the -Property] parameter, the properties that appear in the display depend on the first object's properties. For example, if the first object has PropertyA and PropertyB but subsequent objects have PropertyA, PropertyB, and PropertyC, then only the PropertyA and PropertyB headers will display

Format-Table (Microsoft.PowerShell.Utility)

The Format-Table cmdlet formats the output of a command as a table with the selected properties of the object in each column. The object type determines the default layout and properties that are displayed in each column. You can use the Property parameter to select the properties that you want to display. PowerShell uses default formatters to define how object types are displayed. You can use .ps1xml files to create custom views that display an output table with specified properties. After a custom view is created, use the View parameter to display the table with your custom view. For more information about views, see about_Format.ps1xml. You can use a hash table to add calculated properties to an object before displaying it and to specify the column headings in the table. To add a calculated property, use the Property or GroupBy parameter. For more information about hash tables, see about_Hash_Tables.

[vexx32]

That's Format-Table, sure.

It's also in Export-Csv's documentation here:
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/export-csv?view=powershell-7#notes

When you submit multiple objects to Export-CSV, Export-CSV organizes the file based on the properties of the first object that you submit. If the remaining objects do not have one of the specified properties, the property value of that object is null, as represented by two consecutive commas. If the remaining objects have additional properties, those property values are not included in the file.

Export-Csv (Microsoft.PowerShell.Utility)

The Export-CSV cmdlet creates a CSV file of the objects that you submit. Each object is a row that includes a comma-separated list of the object's property values. You can use the Export-CSV cmdlet to create spreadsheets and share data with programs that accept CSV files as input. Do not format objects before sending them to the Export-CSV cmdlet. If Export-CSV receives formatted objects the CSV file contains the format properties rather than the object properties. To export only selected properties of an object, use the Select-Object cmdlet.

[gabrielsroka]

Ok, fair enough. My PowerShell 5 offline help doesn't mention that.

So, does it make sense for these commands to "default" to using the properties from the first object? Or should it require you to specify which properties you want, so you're not surprised?

[vexx32]

I think allowing users to specify the properties in the cmdlet itself does make some sense, and probably makes a little more sense than trying to search the entire (arbitrarily-large) list for potentially different objects.