Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to detect the encoding of the loaded file before displaying data ? #82

Open
dimitri-hoareau-WEL opened this issue Apr 19, 2021 · 6 comments · May be fixed by #100
Open

How to detect the encoding of the loaded file before displaying data ? #82

dimitri-hoareau-WEL opened this issue Apr 19, 2021 · 6 comments · May be fixed by #100
Assignees

Comments

@dimitri-hoareau-WEL
Copy link

dimitri-hoareau-WEL commented Apr 19, 2021

Hi !

I had a problem for displaying special characters with CVSReader due to csv files which were encoded as ISO-8859-1 instead of utf-8 (ex : Pyr�n�es-Atlantiques )

By adding :

<CSVReader
  config={
    encoding: "ISO-8859-1",
  }
>
 ...
</CSVReader>

in my code it's working, the browser can now read special characters without problems. But the new problem is when loading a file encoded in utf-8, the characters are not displayed properly (ex : Pyrénées-Atlantiques )

My problem is that I am working with clients who do not use the same encoding for their csv files. Some clients use "utf-8", others use "ISO-8859-1". And I can not know in advance what will be the encoding of the file used.

Here is my code :

let changeEncoding = false 
  
   const  handleOnFileLoad = (data) => {
  
    data.map(element => {
      if (element.data.find(element => element.includes("�"))) {
        changeEncoding = true
      } 
    })
    if (changeEncoding) {
      alert("Some characters of your file will not display properly. Please load again yout file.")
      dispatch(setEncodingForExport("ISO-8859-1"))
    } else {
      dispatch(setEncodingForExport("UTF-8"))
    }
    data = data.slice(1)
  
    const enrollFieldArray = {}
  
    enrollFieldArray["data"] = data.map((element => 
      element.data
    ))
  
    dispatch(getDataFromUploadedCsv(enrollFieldArray));
    displayTable();
  
  
  };
  <CSVReader
  onFileLoad={handleOnFileLoad}
  onError={handleOnError}
  ref={buttonRef}
  noClick
  noDrag
  config={encoding}
  >

I use redux and the "encoding" variable is in the state with this default value :

encoding: {encoding: "UTF-8"}

With this solution, the client must load the file a first time to update the state with the correct value of "encoding", and load the file a second time to display data with the correct encoding.

Is there a native CSVReader's method that allows you to detect the encoding of the loaded file before displaying the data ?

Thank you very much for your help

Dimitri

@Bunlong
Copy link
Owner

Bunlong commented Apr 20, 2021

@dimitri-hoareau-WEL Would you like to check data of the file is UTF-8 or ISO-8859-1 before upload?

@Bunlong Bunlong self-assigned this Apr 20, 2021
@dimitri-hoareau-WEL
Copy link
Author

Yes ! I wanted to know if it's possible to do that ?

@luuddan
Copy link

luuddan commented Jun 16, 2021

Hi!

I am having the same issue, is this being worked on?

@exaucae
Copy link

exaucae commented Sep 7, 2021

I may send a pull request in the coming weeks!

@andirkh
Copy link

andirkh commented Oct 13, 2021

@exaucae would you mind sharing any update? I'm having this issue as well

@exaucae exaucae linked a pull request Oct 14, 2021 that will close this issue
@exaucae
Copy link

exaucae commented Oct 14, 2021

@andirkh ,thanks for the bump. Pull request is #100. Feedbacks welcomed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants