Merge master (#4)

* Convert requirements from exact to minimum Fixes swaroopch#40 * Bump version to 0.5.13 * LICENSE: bump year * Fix ambiguous variable name (swaroopch#44) * Add a Changelog * Fix parsing of exact-precision floats with a negative exposant (swaroopch#47) * Accept missing printable ASCII chars (swaroopch#45) * Add a sort_sets optional keyword argument to dump (swaroopch#42) * add failing unit test * add ImmutableList * use ImmutableList on code * fix latin * conform to flake8 * add test * add placeholder file * workaround for limitation on the image downstream * remove conda.txt file * add docstring * descend from Sequence instead of MutableSequence * copy explicitly * remove MutableSequence tests * bump version * fix formatting * use copy.copy() as .copy() is not available for lists on python2 * remove redundant tests * simplify condition * Raise custom exceptions on syntax errors (swaroopch#46) * Add CONTRIBUTING.md (swaroopch#48) * Handle fractions edn-format/edn#64 * Handle #_ Fixes #4. Note it doesn’t support top-level #_ usage such as in: foo #_ bar * Add tests for #_ * Tweak README regarding caveats + mention @bfontaine contributor * Bump version to 0.6.0 Thanks @bfontaine and @konr! * Add v0.6.0 to the Changelog * Run Travis tests on latest Python 3.7 * Use dist: xenial * Use mutable data structures to improve parsing time Previously a list was used to hold the intermediate results during parsing of "expressions", but each expression was prepended by copying, Fixing this takes parsing from quadratic to linear time. * Bump version to 0.6.1 * Update setup.py to 0.6.1 * Various code simplifications (swaroopch#53) * Parse nil and booleans as symbols (swaroopch#54) * Add Vagrantfile for local dev * Use collections.abc instead of collections Fixes swaroopch#55 * Fix build issues https://travis-ci.org/swaroopch/edn_format/jobs/475411817 * Fix Python 2.x compatibility https://travis-ci.org/swaroopch/edn_format/jobs/475413453 * Bump version to 0.6.2 * DRY the requirements Move from setup.py to only requirements.txt * Add badge for PyPI version * add support for unicode char literals * fixed indentation in test_sort_keys * combined version_info checks in edn_format/edn_lex.py Co-Authored-By: LeXofLeviafan <lexofleviafan@gmail.com> * fixed comment in char token definition * made changes according to flake8 demands * Bump version to 0.6.3 * Travis : Upgrade Ubuntu version to latest LTS, bionic * Use tag signing * Add an edn_parse.tag decorator, + docs and tests on tags (swaroopch#59) * Contributor Notes: replace the deprecated commands * Version 0.6.4 * Move contributor notes in CONTRIBUTING.md (swaroopch#61) * Add edn_format.loads_all to parse all expressions in a string (swaroopch#60) * Parse hexadecimal notation Fixes swaroopch#62. * Disallow 0-prefixed integers From the spec: > No integer other than 0 may begin with 0. * Release v0.6.5 * Bump version to 0.6.5-nu
nubank · Nov 25, 2019 · 44c70b0 · 44c70b0
1 parent 116fd10
commit 44c70b0
Show file tree

Hide file tree

Showing 17 changed files with 644 additions and 190 deletions.
diff --git a/.gitignore b/.gitignore
@@ -7,4 +7,5 @@ dist
 edn_format.egg-info/
 *_flymake.py
 venv
-build
+build
+.vagrant
diff --git a/.travis.yml b/.travis.yml
@@ -1,7 +1,8 @@
+dist: bionic  # latest LTS as per https://wiki.ubuntu.com/Releases
 language: python
 python:
   - "2.7"
-  - "3.6"
+  - "3.7"
 install:
   - "pip install -r requirements.txt"
   - "pip install flake8"

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,112 @@
+# `edn_format` Changelog
+
+## v0.6.5 (2019/11/03)
+
+* Parse integers in hexadecimal notation
+* Disallow `0`-prefixed integers other than zero itself
+
+## v0.6.4 (2019/09/14)
+
+* Add an `edn_parse.tag` decorator
+
+## v0.6.3 (2019/04/16)
+
+* Add support for Unicode char literals
+
+## v0.6.2 (2019/01/04)
+
+* Parse `nil` and booleans as symbols
+
+## v0.6.1 (2018/09/21)
+
+* Use mutable data structures to improve parsing time
+* Run Travis tests on Python 3.7
+
+## v0.6.0 (2018/09/08)
+
+* Fix vector parser to use `ImmutableList`
+* Fix parsing of exact-precision floats with a negative exposant
+* Support all ASCII characters
+* Add a `sort_sets` optional argument to `dumps`
+* Raise custom exceptions on syntax errors
+* Support fractions
+* Support `#_`
+
+## v0.5.13 (2017/10/08)
+
+* Convert requirements from exact to minimum
+
+## v0.5.12 (2017/02/14)
+
+* Unbreak tests on Python 2.x
+* PEP8 style fixes
+
+## v0.5.11 (2017/02/14)
+
+* Add a `sort_keys` optional keyword argument to `dumps`
+* Correctly parse floats with an exponent
+
+## v0.5.10 (2017/02/13)
+
+* Support string-\>keyword keys in `dumps`
+* Add Travis config
+* Bump dependencies
+
+## v0.5.9 (2015/07/09)
+
+* Add support to dump `#inst` with microseconds
+
+## v0.5.8 (2015/06/19)
+
+* Fix Python 2/3 support
+
+## v0.5.7 (2015/06/08)
+
+* Fix Python 3 `unichr`/`chr` incompatibility
+* Changed Python version detection from exactly equal to 3 to greater than or
+  equal to 3
+
+## v0.5.6 (2015/05/31)
+
+* Make UTF-8 the default expectation of text type
+
+## v0.5.5 (2015/05/25)
+
+* Fixed string parsing and escaping
+* Unicode now consistently used internally
+* New method `dumpu` added that returns unicode
+* `dumps` method now takes optional encoding arguments: `output_encoding`
+  (specifies encoding of output string) and `string_encoding` (specifies
+  encoding of non-unicode strings in object to be serialized), both default to
+  `'utf-8'`
+* `loads` now takes optional argument `input_encoding`, defaults to `'utf-8'`
+* New method `loadu` added that assumes unicode input
+* Bump dependencies
+
+## v0.5.4 (2015/02/26)
+
+* Fix parsing of booleans before comma
+
+## v0.5.3 (2014/05/03)
+
+## v0.5 (2014/01/25)
+
+* Fix `BaseEdnType.__hash__`
+* Fix equality
+* Handle escape sequences in strings
+* Allow backslashed double-quotes in strings
+* Add `<`, `>`, `@`
+
+## v0.4 (2014/01/24)
+
+* Parse empty collections
+* Raise an exception on syntax error
+* Improves patterns for `SYMBOL`, `TAG`, and `KEYWORD` to more closely match
+  the [EDN specifications][spec]
+* Improve Python 3 support
+
+[spec]: https://github.com/edn-format/edn
+
+## v0.3.6 (2014/01/15)
+
+First published release.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,60 @@
+# Contributing to edn\_format #
+
+## Setup ##
+
+1. Make sure you have a working Python installation (2 or 3). Use either
+   [Virtualenv][], Vagrant, or a similar tool to create an isolated
+   environment:
+    ```bash
+    # 1. One-time: Install Vagrant
+    #
+    # macOS
+    # Install Homebrew from https://brew.sh
+    # brew cask install virtualbox vagrant
+    #
+    # All OSes
+    # Install from https://www.vagrantup.com/downloads.html
+
+    # 2. One-time: Install Vagrant plugin
+    vagrant plugin install vagrant-vbguest
+
+    # 3. This is all you need
+    vagrant up
+
+    # 4. To access the dev environment via ssh
+    vagrant ssh
+    cd /vagrant
+    ```
+
+2. Install `edn_format`’s dependencies: `pip install -r requirements.txt`
+3. Install `flake8`: `pip install flake8`
+
+[Virtualenv]: https://virtualenv.pypa.io/en/stable/#virtualenv
+
+## Tests ##
+
+Run unit tests with:
+
+    python tests.py
+
+Check Python warnings with:
+
+    python -Wall -c 'import edn_format'
+
+Run a linter over the code with:
+
+    flake8 --max-line-length=100 --exclude=parsetab.py .
+
+## Release a new version ##
+
+To release a new version:
+
+1. Ensure you have [setup GPG](https://help.github.com/en/articles/generating-a-new-gpg-key) and [`twine`](https://pypi.org/project/twine/)
+2. Bump up the version number in `setup.py`, e.g. `0.6.3`
+3. Create a git tag: `git tag -s v0.6.3 -m 'Version 0.6.3'` (use [signed tags](https://help.github.com/en/articles/signing-tags))
+4. Verify git tag: `git tag -v v0.6.3`
+5. Push git tag: `git push origin master --tags`
+6. Clean your `dist/` directory if it already exists
+7. Package the release: `python setup.py sdist bdist_wheel`
+8. Check the package: `twine check dist/*`
+9. Upload the package: `twine upload dist/*`
diff --git a/LICENSE.txt b/LICENSE.txt
@@ -1,4 +1,4 @@
-Copyright 2012-2017, Swaroop C H <swaroop@swaroopch.com>
+Copyright 2012-2018, Swaroop C H <swaroop@swaroopch.com>
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

diff --git a/README.md b/README.md
@@ -2,7 +2,10 @@
 
 Implements the [EDN format](https://github.com/edn-format/edn) in Python.
 
+All features of EDN are implemented, including custom tagged elements.
+
 [![Build Status](https://travis-ci.org/swaroopch/edn_format.svg?branch=master)](https://travis-ci.org/swaroopch/edn_format)
+[![PyPI version](https://img.shields.io/pypi/v/edn_format.svg)](https://pypi.org/project/edn_format/)
 
 ## Installation ##
 
@@ -16,6 +19,8 @@ Implements the [EDN format](https://github.com/edn-format/edn) in Python.
 '#{1 2 3}'
 >>> edn_format.loads("[1 true nil]")
 [1, True, None]
+>>> edn_format.loads_all("1 2 3 4")
+[1, 2, 3, 4]
 ```
 
 
@@ -24,29 +29,13 @@ false, it may be a bug.
 
 See `tests.py` for full details.
 
-## Caveats ##
-
-Almost all features of EDN have been implemented, including custom
-tagged elements.
-
-But expect bugs since this has not yet been used in production.
-
 ## Contributors ##
 
 Special thanks to the following contributors for making this library
 usable:
 
+- [@bfontaine](https://github.com/bfontaine)
 - [@marianoguerra](https://github.com/marianoguerra)
 - [@bitemyapp](https://github.com/bitemyapp)
 - [@jashugan](https://github.com/jashugan)
 - [@exilef](https://github.com/exilef)
-
-## Contributor Notes ##
-
-To release a new version:
-
-1. Bump up the version number in `setup.py`, e.g. `0.5.13`
-2. Create a git tag: `git tag -a v0.5.13 -m 'Version 0.5.13'`
-3. Push git tag: `git push origin master --tags`
-4. Make sure you have a [~/.pypirc file](http://docs.python.org/2/distutils/packageindex.html#pypirc) with your PyPI credentials.
-5. Run `python setup.py sdist upload`
diff --git a/Vagrantfile b/Vagrantfile
@@ -0,0 +1,15 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# https://www.vagrantup.com/docs/
+Vagrant.configure("2") do |config|
+  config.vm.box = "ubuntu/cosmic64" # https://cloud-images.ubuntu.com
+
+  config.vm.provider "virtualbox" do |vb|
+    vb.customize ["modifyvm", :id, "--memory", "4096"]
+    vb.name = "edn_format"
+  end
+
+  # https://www.vagrantup.com/docs/provisioning/shell.html
+  config.vm.provision "shell", path: "install.sh", privileged: false, sensitive: true
+end
diff --git a/edn_format/__init__.py b/edn_format/__init__.py
@@ -2,20 +2,24 @@
 from __future__ import absolute_import, division, print_function, unicode_literals
 
 from .edn_lex import Keyword, Symbol
-from .edn_parse import parse as loads
-from .edn_parse import add_tag, remove_tag, TaggedElement
+from .edn_parse import parse as loads, parse_all as loads_all
+from .edn_parse import add_tag, remove_tag, tag, TaggedElement
 from .edn_dump import dump as dumps
+from .exceptions import EDNDecodeError
 from .immutable_dict import ImmutableDict
 from .immutable_list import ImmutableList
 
 __all__ = (
     'ImmutableList',
+    'EDNDecodeError',
     'ImmutableDict',
     'Keyword',
     'Symbol',
     'TaggedElement',
     'add_tag',
     'dumps',
     'loads',
+    'loads_all',
     'remove_tag',
+    'tag',
 )
diff --git a/edn_format/edn_dump.py b/edn_format/edn_dump.py
@@ -3,6 +3,7 @@
 
 import datetime
 import decimal
+import fractions
 import itertools
 import re
 import sys
@@ -62,12 +63,14 @@ def seq(obj, **kwargs):
 def udump(obj,
           string_encoding=DEFAULT_INPUT_ENCODING,
           keyword_keys=False,
-          sort_keys=False):
+          sort_keys=False,
+          sort_sets=False):
 
     kwargs = {
         "string_encoding": string_encoding,
         "keyword_keys": keyword_keys,
         "sort_keys": sort_keys,
+        "sort_sets": sort_sets,
     }
 
     if obj is None:
@@ -89,11 +92,13 @@ def udump(obj,
         return unicode_escape(obj)
     elif isinstance(obj, tuple):
         return '({})'.format(seq(obj, **kwargs))
-    elif isinstance(obj, list):
+    elif isinstance(obj, (list, ImmutableList)):
         return '[{}]'.format(seq(obj, **kwargs))
     elif isinstance(obj, ImmutableList):
         return '[{}]'.format(seq(obj, **kwargs))
     elif isinstance(obj, set) or isinstance(obj, frozenset):
+        if sort_sets:
+            obj = sorted(obj)
         return '#{{{}}}'.format(seq(obj, **kwargs))
     elif isinstance(obj, dict) or isinstance(obj, ImmutableDict):
         pairs = obj.items()
@@ -103,6 +108,8 @@ def udump(obj,
             pairs = ((Keyword(k) if isinstance(k, (bytes, basestring)) else k, v) for k, v in pairs)
 
         return '{{{}}}'.format(seq(itertools.chain.from_iterable(pairs), **kwargs))
+    elif isinstance(obj, fractions.Fraction):
+        return '{}/{}'.format(obj.numerator, obj.denominator)
     elif isinstance(obj, datetime.datetime):
         return '#inst "{}"'.format(pyrfc3339.generate(obj, microseconds=True))
     elif isinstance(obj, datetime.date):
@@ -120,11 +127,13 @@ def dump(obj,
          string_encoding=DEFAULT_INPUT_ENCODING,
          output_encoding=DEFAULT_OUTPUT_ENCODING,
          keyword_keys=False,
-         sort_keys=False):
+         sort_keys=False,
+         sort_sets=False):
     outcome = udump(obj,
                     string_encoding=string_encoding,
                     keyword_keys=keyword_keys,
-                    sort_keys=sort_keys)
+                    sort_keys=sort_keys,
+                    sort_sets=sort_sets)
     if __PY3:
         return outcome
     return outcome.encode(output_encoding)