Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request - compress URLs to better handle a bit more data in the url #294

Open
cheapsteak opened this issue Dec 3, 2020 · 1 comment

Comments

@cheapsteak
Copy link
Contributor

Do you want to request a feature or report a bug?

Feature, related to (but doesn't entirely fix) #34

What is the current behavior?

There seems to be a huge amount of data passed in the URL that does not handle url encoding well,

for example, there's this url, where the post-url-encoded query param is 2419 characters long https://service.bundlewatch.io/results?d=details%7CrepoOwner%7Cbundlewatch%7CrepoName%7CrepoCurrentBranch%7Cdependabot/npm_and_yarn/gzip-size-6.0.0%7CrepoBranchBase%7Cmaster%7CcommitSha%7Cc9ae69ef43d3e4b0ec1661ee13b9ebb7112e2842%7Cresults%7Cstatus%7Cpass%7CfullResults%7CfilePath%7C./__testdata__/test-file-1.jpg%7Csize%7CbaseBranchSize%7CmaxSize%7C./__testdata__/test-file-2.jpg%7C./lib/app/analyze/analyze.test.mockdata.js%7C./lib/app/analyze/analyzeFiles/index.js%7C./lib/app/analyze/index.js%7C./lib/app/config/ensureValid.js%7C./lib/app/config/getCIVars.js%7C./lib/app/config/getConfig.js%7C./lib/app/errors/ValidationError.js%7C./lib/app/getLocalFileDetails/getSize.js%7C./lib/app/getLocalFileDetails/index.js%7C./lib/app/index.js%7C./lib/app/reporting/BundleWatchService/index.js%7C./lib/app/reporting/GitHubService/index.js%7C./lib/app/resultsPage/createURL.js%7C./lib/app/resultsPage/shortenURL.js%7C./lib/bin/determineConfig.js%7C./lib/bin/index.js%7C./lib/logger/index.js%7C./artifacts/bundlewatch-0.0.0.tgz%7Csummary%7CEverything%2Bis%2Bin%2Bcheck%2B(%252B2B,%2B-0B,%2B%252B0.0%2525)%5EU2A%7CU2A%7CAZ28%7C5GKT%7C5GKT%7CAZ28%7C6L%7C6L%7C1KW%7CPO%7CPO%7C1KW%7CYY%7CYY%7C1KW%7CWB%7CWB%7C1KW%7CH1%7CH1%7C1KW%7CFR%7CFR%7C1KW%7C5A%7C5A%7C1KW%7CGC%7CGC%7C1KW%7CJT%7CJT%7C1KW%7C120%7C120%7C1KW%7COE%7COE%7C1KW%7CS4%7CS4%7C1KW%7CF1%7CF1%7C1KW%7CFZ%7CFZ%7C1KW%7CQX%7CQX%7C1KW%7C14V%7C14V%7C1KW%7CFP%7CFP%7C1KW%7CB5A%7CB58%7CBUO%5E%5E$0%7C$1%7C2%7C3%7C2%7C4%7C5%7C6%7C7%7C8%7C9%5D%7CA%7C$B%7CC%7CD%7C@$E%7CF%7CB%7CC%7CG%7C14%7CH%7C15%7CI%7C16%5D%7C$E%7CJ%7CB%7CC%7CG%7C17%7CH%7C18%7CI%7C19%5D%7C$E%7CK%7CB%7CC%7CG%7C1A%7CH%7C1B%7CI%7C1C%5D%7C$E%7CL%7CB%7CC%7CG%7C1D%7CH%7C1E%7CI%7C1F%5D%7C$E%7CM%7CB%7CC%7CG%7C1G%7CH%7C1H%7CI%7C1I%5D%7C$E%7CN%7CB%7CC%7CG%7C1J%7CH%7C1K%7CI%7C1L%5D%7C$E%7CO%7CB%7CC%7CG%7C1M%7CH%7C1N%7CI%7C1O%5D%7C$E%7CP%7CB%7CC%7CG%7C1P%7CH%7C1Q%7CI%7C1R%5D%7C$E%7CQ%7CB%7CC%7CG%7C1S%7CH%7C1T%7CI%7C1U%5D%7C$E%7CR%7CB%7CC%7CG%7C1V%7CH%7C1W%7CI%7C1X%5D%7C$E%7CS%7CB%7CC%7CG%7C1Y%7CH%7C1Z%7CI%7C20%5D%7C$E%7CT%7CB%7CC%7CG%7C21%7CH%7C22%7CI%7C23%5D%7C$E%7CU%7CB%7CC%7CG%7C24%7CH%7C25%7CI%7C26%5D%7C$E%7CV%7CB%7CC%7CG%7C27%7CH%7C28%7CI%7C29%5D%7C$E%7CW%7CB%7CC%7CG%7C2A%7CH%7C2B%7CI%7C2C%5D%7C$E%7CX%7CB%7CC%7CG%7C2D%7CH%7C2E%7CI%7C2F%5D%7C$E%7CY%7CB%7CC%7CG%7C2G%7CH%7C2H%7CI%7C2I%5D%7C$E%7CZ%7CB%7CC%7CG%7C2J%7CH%7C2K%7CI%7C2L%5D%7C$E%7C10%7CB%7CC%7CG%7C2M%7CH%7C2N%7CI%7C2O%5D%7C$E%7C11%7CB%7CC%7CG%7C2P%7CH%7C2Q%7CI%7C2R%5D%5D%7C12%7C13%5D%5D
calling decodeURIComponent on that reveals the original value is 1719 characters long (so URL encoding inflated it by 40%) details|repoOwner|bundlewatch|repoName|repoCurrentBranch|dependabot/npm_and_yarn/gzip-size-6.0.0|repoBranchBase|master|commitSha|c9ae69ef43d3e4b0ec1661ee13b9ebb7112e2842|results|status|pass|fullResults|filePath|./__testdata__/test-file-1.jpg|size|baseBranchSize|maxSize|./__testdata__/test-file-2.jpg|./lib/app/analyze/analyze.test.mockdata.js|./lib/app/analyze/analyzeFiles/index.js|./lib/app/analyze/index.js|./lib/app/config/ensureValid.js|./lib/app/config/getCIVars.js|./lib/app/config/getConfig.js|./lib/app/errors/ValidationError.js|./lib/app/getLocalFileDetails/getSize.js|./lib/app/getLocalFileDetails/index.js|./lib/app/index.js|./lib/app/reporting/BundleWatchService/index.js|./lib/app/reporting/GitHubService/index.js|./lib/app/resultsPage/createURL.js|./lib/app/resultsPage/shortenURL.js|./lib/bin/determineConfig.js|./lib/bin/index.js|./lib/logger/index.js|./artifacts/bundlewatch-0.0.0.tgz|summary|Everything+is+in+check+(%2B2B,+-0B,+%2B0.0%25)^U2A|U2A|AZ28|5GKT|5GKT|AZ28|6L|6L|1KW|PO|PO|1KW|YY|YY|1KW|WB|WB|1KW|H1|H1|1KW|FR|FR|1KW|5A|5A|1KW|GC|GC|1KW|JT|JT|1KW|120|120|1KW|OE|OE|1KW|S4|S4|1KW|F1|F1|1KW|FZ|FZ|1KW|QX|QX|1KW|14V|14V|1KW|FP|FP|1KW|B5A|B58|BUO^^$0|$1|2|3|2|4|5|6|7|8|9]|A|$B|C|D|@$E|F|B|C|G|14|H|15|I|16]|$E|J|B|C|G|17|H|18|I|19]|$E|K|B|C|G|1A|H|1B|I|1C]|$E|L|B|C|G|1D|H|1E|I|1F]|$E|M|B|C|G|1G|H|1H|I|1I]|$E|N|B|C|G|1J|H|1K|I|1L]|$E|O|B|C|G|1M|H|1N|I|1O]|$E|P|B|C|G|1P|H|1Q|I|1R]|$E|Q|B|C|G|1S|H|1T|I|1U]|$E|R|B|C|G|1V|H|1W|I|1X]|$E|S|B|C|G|1Y|H|1Z|I|20]|$E|T|B|C|G|21|H|22|I|23]|$E|U|B|C|G|24|H|25|I|26]|$E|V|B|C|G|27|H|28|I|29]|$E|W|B|C|G|2A|H|2B|I|2C]|$E|X|B|C|G|2D|H|2E|I|2F]|$E|Y|B|C|G|2G|H|2H|I|2I]|$E|Z|B|C|G|2J|H|2K|I|2L]|$E|10|B|C|G|2M|H|2N|I|2O]|$E|11|B|C|G|2P|H|2Q|I|2R]]|12|13]]
Using lz compression (`lzString.compressToEncodedURIComponent(input)`), the resulting string has a length of 1433 characters (16% reduction) CYUwLghglgNgzgHwE4gA4HsDyB3AdiJBAIwFddgYRsIwBjAC2TXQDkIBbEJjAYRKRS4wAISQRcDBKFQhyEIujAB6XKnYB9ccHUBPCElxKA5gC8oqALRwoJkBYBsAOgAML7ulHiGwiHC7tfMAIEWnR2digwAGV6CBCATggQe3iQADMAFgBmYCyQDKJnEFoARnt7EpAQEqyiVKIiAHYSkoAmEFaADgzWpjgSGDBEOEgwEkRUX0Q0gZgAJRB+wenYEAAFGkZHJXV1IJHgGghdpX2wCzTVixLHACtUIwRrW2JfEE8JeiibfwgAD2+L22uzOh0gJzOFyurTuDwQ2xgUCISggqFQKNwEBgOlsGKxOJAjjOjnY6FoAGswRA7ogEUiUWi8djceJ8bYAGKrOBKKDkEB-GnwpSI5Go9Gs5kgHl8gW3WnC+lipShXCXIxKWT9FAANSxUGAgrposZKrVxnAPAAkrqkHBDQrjejTVB1UYLehVS77SKGeiCEh0LalLrEWCoB6AKICQPexWMt1gAAyZKxnMoABFwNB4Oboj9Y47c8naKnVpnILBubzQLL5T6ldX+QXfUoUBgkGBeerhGQKCAAOo0BhRAgANygtCljdrQvrjLbgc7uHVAHFIgAJEhEEdIceT6U15tKlBLIYbN3KlA0EAAVTmiaP88WAzPEAvcHoi9kd4fctn9KIXklFAIIkAifAeA9NVm0Awxp2bGB0CMN0kAPJs-22fROzSCBaCGJRSHIShqDoegLFcCiiVMJ4SHCfQdAQCNRwIHQwHoLsAGooDgLjcA4hhinJDiAAoAFJWmECSABoOPI4QZPE4QKPEgBWABKAA9G9WgAQQQbS9J0gAtLoEBUlcAGkABUzMsmzjNM+xEwQJyEBKCz+wQNZMC8nz3M8gBNAKECCtyPIQfthAiqL-IQdcSjihLYvZOYEBSsLPJUvSsoyhAVx4PKCtigApGzStytpnDc1oqtizAIwQercqiDIEBa3L2QSzqOqMtLetigBFAANBBhoqjJtTciaOrWNLZti4QcsWzoEGEG9MA0jSABIqq2hLeiyBBelalSXIQRoEBW+IAF0ED0raooK9MEAAAS2hr2VWhACpXKa4rc07LTc+xbvehBiq+n63Iu9c3JWoGShuhAwYsyG8rcvTYZKKKEZ4UGGucx70ZKZ6sYahH2XxhAAFk0d+kpfqx2GEctKmWDptyIax1GEcTKmfKJ+naax9mEcwKnZsFtzZqxgaEARuYqblqWSiif6ShshGbyp1KVcmrHPIRoaqbVlXgqx3qgZqqmbKl1oEth1peitrIqZvDnWlax3AaOkHkYaya7Zho74aOpGwc8u3MaOnGjrx-2EBGu3SaO8mjsphPgrtxmjuZo7WYT3q7a5o6eaOvmE5KKq7eFo7RaO8XK4Su2ZaOuWrcV262jc13rqAA

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?

If this is a feature request, what is motivation or use case for changing the behavior?

Long urls are breaking the details page (getting 494 Request header too large errors trying to view them);

Compressing the query params into a format that doesn't get inflated by url encoding would add a bit more headroom to how long a URL can be without breaking the link.

It doesn't solve it completely though (using Checks API to publish the table to the repo as a markdown summary would #293 but that might be a larger change / take more time)

Please mention other relevant information such as the browser version, Node.js version, bundlewatch version, and Operating System.

@cesargdm
Copy link

For my Create React App I'm getting

Screen Shot 2022-01-12 at 11 23 03 p m

Unable to shorten URL code=Request failed with status code 400

And a wired huge URL (trimmed, but it goes for over 200 characters): https://service.bundlewatch.io/results?d=details%7CrepoOwner%7C<OMMITED>%7CrepoName%7C<OMMITED>%7CrepoCurrentBranch%7Cdevelop%7CrepoBranchBase%7CcommitSha%7C20ef493aff96f1bb129a2b6f0ca5fc75c18c91bf%7Cresults%7Cstatus%7Cpass%7CfullResults%7CfilePath%7Cbuild%2Fservice-worker.js%7Csize%7CbaseBranchSize%7CmaxSize%7Cbuild...

My error appears to be related to #293

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants