Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

url field is having its special characters escaped in BBT Bibtex #1716

Closed
rmoraisr opened this issue Jan 3, 2021 · 8 comments
Closed

url field is having its special characters escaped in BBT Bibtex #1716

rmoraisr opened this issue Jan 3, 2021 · 8 comments
Labels

Comments

@rmoraisr
Copy link

rmoraisr commented Jan 3, 2021

Exporter used: Better Bibtex

Expected behavior:

@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}

Actual behavior:

@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}

The error is in the url field. Although the url field is marked as a verbatim field, when exporting using BBT Bibtex, special characters are escaping (in the example "%" is being escaped).
The error only happens in BBT Bibtex, using BBT Biblatex or the standard Zotero Bibtex, everything is fine.

url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d} with % being scaped.

Version of the systems where the error occurs:
Zotero version: 5.0.94
Zotero version: 5.0.95-beta.3 + 4db61b8dc
OS: macOS 11.1

BetterBibtex Debug Json file:
{
  "collections": {},
  "config": {
    "id": "36a3b0b5-bad0-4a04-b79b-441c7cef77db",
    "label": "BetterBibTeX JSON",
    "localeDateOrder": "dmy",
    "options": {
      "Normalize": false,
      "exportFileData": false,
      "exportNotes": true,
      "keepUpdated": false
    },
    "preferences": {
      "DOIandURL": "both",
      "ascii": "",
      "asciiBibLaTeX": false,
      "asciiBibTeX": true,
      "autoAbbrev": false,
      "autoAbbrevStyle": "",
      "autoExport": "immediate",
      "autoExportDelay": 1,
      "autoExportIdleWait": 10,
      "autoExportPathReplaceDiacritics": false,
      "autoExportPathReplaceDirSep": "-",
      "autoExportPathReplaceSpace": " ",
      "autoPinDelay": 0,
      "automaticTags": true,
      "auxImport": false,
      "biblatexExtendedDateFormat": false,
      "biblatexExtendedNameFormat": false,
      "biblatexExtractEprint": true,
      "bibtexParticleNoOp": false,
      "bibtexURL": "url",
      "cacheFlushInterval": 5,
      "citeCommand": "cite",
      "citekeyFold": true,
      "citekeyFormat": "[auth][shorttitle2_2][year]",
      "citeprocNoteCitekey": false,
      "csquotes": "",
      "debugLogDir": "",
      "exportBibTeXStrings": "off",
      "exportBraceProtection": true,
      "exportTitleCase": true,
      "extraMergeCSL": true,
      "extraMergeCitekeys": true,
      "extraMergeTeX": true,
      "git": "config",
      "ignorePostscriptErrors": true,
      "import": true,
      "importBibTeXStrings": true,
      "importCaseProtection": "as-needed",
      "importCitationKey": true,
      "importExtra": true,
      "importJabRefAbbreviations": true,
      "importJabRefStrings": true,
      "importSentenceCase": "on+guess",
      "itemObserverDelay": 5,
      "jabrefFormat": 0,
      "keyConflictPolicy": "keep",
      "keyScope": "library",
      "kuroshiro": false,
      "mapMath": "",
      "mapText": "",
      "mapUnicode": "conservative",
      "newTranslatorsAskRestart": true,
      "parseParticles": true,
      "platform": "mac",
      "postscript": "",
      "postscriptOverride": "",
      "qualityReport": true,
      "quickCopyMode": "latex",
      "quickCopyPandocBrackets": false,
      "rawImports": false,
      "rawLaTag": "#LaTeX",
      "relativeFilePaths": false,
      "retainCache": false,
      "scrubDatabase": false,
      "skipFields": "keywords, file,  timestamp",
      "skipWords": "a,ab,aboard,about,above,across,after,against,al,along,amid,among,an,and,anti,around,as,at,before,behind,below,beneath,beside,besides,between,beyond,but,by,d,da,das,de,del,dell,dello,dei,degli,della,dell,delle,dem,den,der,des,despite,die,do,down,du,during,ein,eine,einem,einen,einer,eines,el,en,et,except,for,from,gli,i,il,in,inside,into,is,l,la,las,le,les,like,lo,los,near,nor,of,off,on,onto,or,over,past,per,plus,round,save,since,so,some,sur,than,the,through,to,toward,towards,un,una,unas,under,underneath,une,unlike,uno,unos,until,up,upon,versus,via,von,while,with,within,without,yet,zu,zum",
      "strings": "",
      "testing": false,
      "verbatimFields": "url,doi,file,eprint,verba,verbb,verbc,groups",
      "warnBulkModify": 1,
      "warnTitleCased": false,
      "workers": 0
    }
  },
  "items": [
    {
      "abstractNote": "Em aprendizagem de máquina, a medição de desempenho é uma tarefa crucial. As métricas ROC (Receiver Operating Characteristics) e AUC (Area…",
      "accessDate": "2020-11-23",
      "attachments": [
        {
          "dateAdded": "2020-12-31T15:41:19Z",
          "dateModified": "2020-12-31T15:41:19Z",
          "itemType": "attachment",
          "path": "/Users/rflmorais/Zotero/storage/JER6PBMW/o-que-é-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d.html",
          "relations": [],
          "tags": [],
          "title": "Snapshot",
          "uri": "http://zotero.org/users/7315465/items/JER6PBMW"
        }
      ],
      "citationKey": "AvelarQueAUC2019",
      "citekey": "AvelarQueAUC2019",
      "creators": [
        {
          "creatorType": "author",
          "firstName": "Adriano",
          "lastName": "Avelar"
        }
      ],
      "date": "22/09/2020",
      "dateAdded": "2020-12-31T15:41:14Z",
      "dateModified": "2021-01-03T14:17:34Z",
      "extra": "Citation Key: AvelarQueAUC2019",
      "itemID": 71,
      "itemType": "webpage",
      "key": "ZMDN7VRS",
      "language": "pt-BR",
      "libraryID": 1,
      "notes": [],
      "relations": [],
      "select": "zotero://select/library/items/ZMDN7VRS",
      "tags": [],
      "title": "O que é AUC e ROC nos modelos de Machine Learning",
      "uri": "http://zotero.org/users/7315465/items/ZMDN7VRS",
      "url": "https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d",
      "version": 255
    }
  ],
  "version": {
    "bbt": "5.2.102",
    "zotero": "5.0.95-beta.3+4db61b8dc"
  }
}
@label-gun
Copy link

label-gun bot commented Jan 3, 2021

It looks like you did not upload an debug report. The debug report is important; it gives @retorquere your current BBT settings and a copy of the problematic reference as a test case so he can best replicate your problem. Without it, @retorquere is effectively blind. Debug reports are useful for both bug analysis and enhancement requests; in the case of export enhancements, I need the copy of the references you have in mind.

If you did try to submit a debug report, but the ID looked like D<number>, that is a Zotero debug report, which I cannot access. Please re-submit a BBT debug log by one of the methods below.

This request is much more likely than not to apply to you, too, even if you think it unlikely, and even if it does not, there's no harm in sending a debug log that turns out to be unnecessary. @retorquere will more often than not just end up saying "please send a debug log first". Let's just skip over the unnecesary delay this entails. Sending a debug log is very easy:

  1. If your issue relates to how BBT behaves around a specific reference(s), such as citekey generation or export, select at least one of the problematic reference(s), right-click it, and submit an BBT debug report from that popup menu. If the problem is with export, please do include a sample of what you see exported, and what you expected to see exported for these references.

  2. If the issue does not relate to references and is of a more general nature, generate an debug report by restarting Zotero with debugging enabled (Help -> Debug Output Logging -> Restart with logging enabled), reproducing your problem, and selecting "Send Better BibTeX debug report..." from the help menu.

Once done, you will see a debug ID in red. Please post that debug id in the issue here.

Thank you!

@retorquere
Copy link
Owner

When I compile this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-\%C3\%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

I get the URL as expected.

When I compile this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

I get an error, and no URL is output. So it looks like for bibtex, the escaping is necessary.

BTW, if you send a debug log via the steps described in point 1, I get sent the reference in debug log format automatically; my workflow is built around that. This worked though.

@rmoraisr
Copy link
Author

rmoraisr commented Jan 4, 2021

Sorry about the debug stuff. Lesson learned.
Thanks for the quick response. I thought it was a bug in Zotero considering that is a verbatim field. But it seems is not. Im using a flavored bibtex from my university and I'm getting an error with the url with escape chars. I'm suspecting they using some old macros to parse the bib file

Thanks a lot

@retorquere
Copy link
Owner

No worries about the debug log -- I'm happy you sent one. You wouldn't believe how stubborn people can be about sending them. It's one of the primary reasons I built @label-gun, because it would be practically a given people would not send one no matter how I pleaded.

WRT the URL -- things are more complicated than I had thought. It is true that

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

gives me an error, but

\documentclass{article}
\usepackage[utf8]{inputenc}
\bibliographystyle{plainnat}
\usepackage{url} % <------------------- this right here
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@misc{AvelarQueAUC2019,
  title = {{O que \'e AUC e ROC nos modelos de Machine Learning}},
  author = {Avelar, Adriano},
  year = {2020},
  month = sep,
  url = {https://medium.com/@eam.avelar/o-que-%C3%A9-auc-e-roc-nos-modelos-de-machine-learning-2e2c4112033d},
  urldate = {2020-11-23},
  abstract = {Em aprendizagem de m\'aquina, a medi\c{c}\~ao de desempenho \'e uma tarefa crucial. As m\'etricas ROC (Receiver Operating Characteristics) e AUC (Area\ldots},
  language = {pt-BR}
}
\end{filecontents}

\begin{document}

\nocite{*}

\bibliography{\jobname.bib}

\end{document}

compiles just fine. I'll have to think about how to deal with both situations.

@github-actions
Copy link

github-actions bot commented Jan 4, 2021

🤖 this is your friendly neighborhood build bot announcing test build 5.2.102.247 ("fixes #1716")

Install in Zotero by downloading test build 5.2.102.247, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

@rmoraisr
Copy link
Author

rmoraisr commented Jan 4, 2021

Problem solved. It's worked perfectly.
Thanks a lot. It's save me a lot of time

@retorquere
Copy link
Owner

Cool, thanks for the confirmation. A new release is building.

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants