Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.gitattributes File Type Setting can introduce encoding errors in Output/RequiredModules #138

Open
ChristophHannappel opened this issue May 13, 2022 · 0 comments

Comments

@ChristophHannappel
Copy link
Contributor

ChristophHannappel commented May 13, 2022

I had a wired encoding bug with some .psm1 Files in the Output/RequiredModules directory if i had to reinclude them into the repository.
After committing those files the content got scrambled with invisible and strange Unicode characters which broke their functionality
This happened sometimes but not always. In my case the issue was with the DSCBuildHelpers Module.

I found out that the issue only happens if the PowerShell File has UTF16le encoding and the file gets treated as text by git.
Currently the .gitattributes file declares everything as text

* text eol=autocrlf

and overwrites this for some file extensions.
As far as i understand git uses UTF8 internally and when you commit a file which is configured as text it converts it. After that you'll see the encoding errors.
The git default behaviour for a UTF16le file is to treat it as a binary and that way it stays as it should be.
Other PowerShell files encodings are recognised as text.

Why is this an issue
Sometimes you need a custom version of a required module or you have a niche Azure Dev Ops Server Configuration where you can not use the Azure Dev Ops Artifactory Feed.

Proposal
Remove the everything is text declaration or restrict it to the parts where we are in encoding control like /source/** text eol=autocrlf

Repro Steps

  1. Create a PowerShell File with UTF16 le encoding.
  2. Commit the file
  3. Push the commit
  4. Git clone the repository to a different location
  5. Change the file contents at the different location
  6. Commit and push again
  7. Pull at the original location
raandree added a commit to raandree/DscWorkshop that referenced this issue Dec 4, 2022
raandree added a commit that referenced this issue Dec 5, 2022
* Update azure-pipelines.yml for Azure Pipelines

* Update azure-pipelines.yml for Azure Pipelines

* Fix according to Sampler 0.116.0

* Changed autocrlf scope as suggested in #138
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant