Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: default charset to UTF-8 for text/csv if not specified #1423

Merged
merged 2 commits into from Aug 11, 2021
Merged

fix: default charset to UTF-8 for text/csv if not specified #1423

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
052fabe
Fix: Defaulting to UTF-8 if charset is missing
rohitvvv Aug 5, 2021
d32902f
Fix: Defaulting to UTF-8 if charset is missing
rohitvvv Aug 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions google-http-client/src/main/java/com/google/api/client/http/HttpResponse.java
View file
Open in desktop
Expand Up @@ -534,6 +534,10 @@ public Charset getContentCharset() {
// https://tools.ietf.org/html/rfc4627 - JSON must be encoded with UTF-8
return StandardCharsets.UTF_8;
}
// fallback to well-kown charset for text/csv
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
// fallback to well-kown charset for text/csv
// fallback to well-kown charset for text/csv

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if ("text".equals(mediaType.getType()) && "csv".equals(mediaType.getSubType())) {
return StandardCharsets.UTF_8;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we add a comment with the link to the spec in case someone browsing the code wonders why this is here (without having to do a git blame and track down this PR through history).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}
}
return StandardCharsets.ISO_8859_1;
}
Expand Down
28 changes: 28 additions & 0 deletions google-http-client/src/test/java/com/google/api/client/http/HttpResponseTest.java
View file
Open in desktop
Expand Up @@ -68,6 +68,8 @@ public void testParseAsString_none() throws Exception {
private static final String VALID_CONTENT_TYPE = "text/plain";
private static final String VALID_CONTENT_TYPE_WITH_PARAMS =
"application/vnd.com.google.datastore.entity+json; charset=utf-8; version=v1; q=0.9";
private static final String VALID_CONTENT_TYPE_WITHOUT_CHARSET =
"text/csv; version=v1; q=0.9";
private static final String INVALID_CONTENT_TYPE = "!!!invalid!!!";
private static final String JSON_CONTENT_TYPE = "application/json";

Expand Down Expand Up @@ -194,6 +196,32 @@ public LowLevelHttpResponse execute() throws IOException {
assertEquals("ISO-8859-1", response.getContentCharset().name());
}

public void testParseAsString_validContentTypeWithoutCharSetWithParams() throws Exception {
HttpTransport transport =
new MockHttpTransport() {
@Override
public LowLevelHttpRequest buildRequest(String method, String url) throws IOException {
return new MockLowLevelHttpRequest() {
@Override
public LowLevelHttpResponse execute() throws IOException {
MockLowLevelHttpResponse result = new MockLowLevelHttpResponse();
result.setContent(SAMPLE2);
result.setContentType(VALID_CONTENT_TYPE_WITHOUT_CHARSET);
return result;
}
};
}
};
HttpRequest request =
transport.createRequestFactory().buildGetRequest(HttpTesting.SIMPLE_GENERIC_URL);

HttpResponse response = request.execute();
assertEquals(SAMPLE2, response.parseAsString());
assertEquals(VALID_CONTENT_TYPE_WITHOUT_CHARSET, response.getContentType());
assertNotNull(response.getMediaType());
assertEquals("UTF-8", response.getContentCharset().name());
}

public void testParseAsString_jsonContentType() throws IOException {
HttpTransport transport =
new MockHttpTransport() {
Expand Down