/
atom.xml
277 lines (242 loc) · 55.8 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Thilo Michael</title><link href="http://uhlo.github.io/" rel="alternate"></link><link href="http://uhlo.github.io/atom.xml" rel="self"></link><id>http://uhlo.github.io/</id><updated>2018-04-25T15:15:00+02:00</updated><subtitle></subtitle><entry><title>A Deep Dive into rasa</title><link href="http://uhlo.github.io/a-deep-dive-into-rasa.html" rel="alternate"></link><published>2018-04-25T15:15:00+02:00</published><updated>2018-04-25T15:15:00+02:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2018-04-25:/a-deep-dive-into-rasa.html</id><summary type="html"><p>With the rise of personal assistants like Siri, Google Assistant, Cortana, Alexa and how they're all called, there seems to be a great interest in chat bots, which are basically small text-based Alexas! So - exactly like 20 years ago - many companies want cool chat-bot interfaces for Facebook, WhatsApp or simply …</p></summary><content type="html"><p>With the rise of personal assistants like Siri, Google Assistant, Cortana, Alexa and how they're all called, there seems to be a great interest in chat bots, which are basically small text-based Alexas! So - exactly like 20 years ago - many companies want cool chat-bot interfaces for Facebook, WhatsApp or simply their website. That's where rasa comes into play. rasa is a startup that provides a framework for building bots utilizing the hottest and newest approaches, including <em>Machine Learning</em>™.</p>
<!-- more -->
<h2>What is rasa?</h2>
<p>That is a difficult question. It is the startup, the "<em>platform</em>", the "<em>stack</em>", the framework. So basically it is piece of software that you can use to build chat-bots and also a company that uses that framework and other tools (as far as I understood it) to build chat-bots for companies.</p>
<p><a href="https://rasa.com"><img alt="A screenshot of rasa.com" src="images/rasa_screenshot.png"></a></p>
<p>The software itself, which is what I will focus on in this blog entry, is split into two projects: <strong>rasa core</strong> and <strong>rasa nlu</strong>.</p>
<p><strong>rasa nlu</strong> handles the natural language understanding. It takes the sentences typed by the users, classifies it into one of several <em>intents</em> (i.e. what is the user intending to do?) and detects <em>entities</em> that are mentioned. So for example, when a user asks: "<em>What is the weather going to be tomorrow in Berlin?</em>" a well trained NLU could return the intent <code>weather_request</code> with the entities <code>date: "tomorrow"</code> and <code>location: "Berlin"</code>. rasa nlu uses <a href="https://spacy.io">spacy</a> under the hood and does its job very well (in my experience).</p>
<p>The second part, <strong>rasa core</strong>, is what I will focus on in this blog post. It is basically "<em>the rest</em>" (which is probably why it's called <em>core</em>) and it is mainly to solve the task "<em>What does the system should say/do given the dialogue up until now</em>". For this, it uses a Recurrent Neural Network that gets the history of the last actions that were taken by the user as well as the system and predicts what action should be taken based on that.</p>
<p>Just to show you how hip this framework is, let me tell you that the training data (dialogues that are used for the RNN to train on) are stored in a Markdown file and the domain is specified in YAML. But in all seriousness, this helps to remove the barrier for newcomers that want to build their own bot.</p>
<p>I recommend you to at least look at their <a href="https://core.rasa.com/tutorial_basics.html">basic tutorial</a>, where you can see that you don't have to write a single line of code to build a (very basic) bot. I think that is very impressive (of course if you want a bot that actually does something, it gets slightly more complex).</p>
<h2>How does it work?</h2>
<p>Damn. This is also a difficult question. In principle, the tutorials on the rasa website try to let you create bots by only showing you the parts that you really need to know. Which is good if you just want to build a bot quickly, but is not good for the deeper understanding.</p>
<p>Throughout the tutorials one page is always linked when it gets to the interesting "under the hood" stuff. The page is titled "Plumbing - How it all fits together" and mostly consists of this image:</p>
<p><a href="https://core.rasa.com/plumbing.html"><img alt="The &quot;Plumbing&quot; of rasa" src="https://core.rasa.com/_images/rasa_arch_colour.png"></a></p>
<p>Sadly, this is not that informative, but let me still try to explain this image a little bit. Basically, chat-bots are a pipeline: message comes in, bot does stuff, message comes out. In this case, the message arrives at the Interpreter <code>1</code>. The Interpreter is rasa nlu and as I described above, it converts the text into something meaningful for the computer, namely an <em>intent</em> and <em>entities</em>.</p>
<p>This information is then handed to the Tracker <code>2</code>. The Tracker is basically the control unit of the chat bot. It keeps track of what the system and user has already said, what information was given by the user (<em>slots</em> that are filled). The Tracker takes the new dialogue act together with the acts from the last few turns and hands them to the Policy <code>3</code>. The Policy is the RNN that then determines what Action <code>4</code> the bot should take. The Action does its thing (for example retrieving the weather from an API), updates the Tracker (so that the Tracker knows which Action was being executed and to update the state accordingly) and sends a message to the user <code>6</code>.</p>
<p>That seems very simple and bots with only a few stories (i.e. training data) already work rather well. So, that's it, right?</p>
<h2>What's a Memoization?</h2>
<p>Digging deeper into the rasa core source code, I saw that this image is oversimplifying (duh). In the second tutorial I found the following code:</p>
<div class="highlight"><pre><span></span><code><span class="n">agent</span> <span class="o">=</span> <span class="n">Agent</span><span class="p">(</span><span class="n">domain_file</span><span class="p">,</span> <span class="n">policies</span><span class="o">=</span><span class="p">[</span><span class="n">MemoizationPolicy</span><span class="p">(),</span> <span class="n">RestaurantPolicy</span><span class="p">()])</span>
</code></pre></div>
<p>This code is executed during training and it seems the the agent is given a list of policies. The <code>RestaurantPolicy</code> is a policy created in the tutorial and is basically an LSTM RNN. The <code>MemoizationPolicy</code> however is not really explained. Here is what the documentation says:</p>
<p><a href="https://core.rasa.com/tutorial_supervised.html#a-custom-dialogue-policy"><img alt="Note: Remember, you do not need to create your own policy. The default policy setup using a memoization policy and a Keras policy works quite well. Nevertheless, you can always fine tune them for your use case. Read Plumbing - How it all fits together for more info." src="images/plumbing.png"></a></p>
<p>The link to the Plumbing page unsurprisingly did not yield new insights. However, the picture with the one policy is not quite complete.</p>
<p>Looking at the source code for the tracker I could find the <code>PolicyEnsemble</code>, a class that can incorporate different policies (i.e. decision makers). So, at each step of the dialogue multiple policies are executed and the best one gets the bid.</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">SimplePolicyEnsemble</span><span class="p">(</span><span class="n">PolicyEnsemble</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">policies</span><span class="p">,</span> <span class="n">known_slot_events</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="nb">super</span><span class="p">(</span><span class="n">SimplePolicyEnsemble</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">policies</span><span class="p">,</span> <span class="n">known_slot_events</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">probabilities_using_best_policy</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tracker</span><span class="p">,</span> <span class="n">domain</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">decision_maker</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">max_confidence</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
<span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">policies</span><span class="p">:</span>
<span class="n">probabilities</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="n">predict_action_probabilities</span><span class="p">(</span><span class="n">tracker</span><span class="p">,</span> <span class="n">domain</span><span class="p">)</span>
<span class="n">confidence</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">probabilities</span><span class="p">)</span>
<span class="k">if</span> <span class="n">confidence</span> <span class="o">&gt;</span> <span class="n">max_confidence</span><span class="p">:</span>
<span class="n">max_confidence</span> <span class="o">=</span> <span class="n">confidence</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">probabilities</span>
<span class="n">decision_maker</span> <span class="o">=</span> <span class="n">p</span>
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">%s</span><span class="s2"> made the decision!&quot;</span> <span class="o">%</span> <span class="n">decision_maker</span><span class="p">)</span>
<span class="k">return</span> <span class="n">result</span>
</code></pre></div>
<p>The <code>SimplePolicyEnsemble</code> asks each policy for their probabilities given the tracker (i.e. the current state) and the domain and selects the decision of the policy with the highest confidence. Note that I added a debug output (<em>decision_maker</em>) which prints the policy that had the highest confidence, just so I could see in each turn which policy was responsible for the actions of the bot.</p>
<p>Let's see what action probabilities the <code>MemoizationPolicy</code> is producing:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">predict_action_probabilities</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">tracker</span><span class="p">,</span> <span class="n">domain</span><span class="p">):</span>
<span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">featurize</span><span class="p">(</span><span class="n">tracker</span><span class="p">,</span> <span class="n">domain</span><span class="p">)</span>
<span class="n">tracker_state</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">featurizer</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="n">x</span><span class="p">,</span>
<span class="n">domain</span><span class="o">.</span><span class="n">input_features</span><span class="p">)]</span>
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s1">&#39;Current tracker state [</span><span class="se">\n\t</span><span class="si">{}</span><span class="s1">]&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="s2">&quot;</span><span class="se">\n\t</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">tracker_state</span><span class="p">)))</span>
<span class="n">memorised</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">recall</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">domain</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.0</span><span class="p">]</span> <span class="o">*</span> <span class="n">domain</span><span class="o">.</span><span class="n">num_actions</span>
<span class="k">if</span> <span class="n">memorised</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">is_enabled</span><span class="p">:</span>
<span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Used memorised next action &#39;</span><span class="si">{}</span><span class="s2">&#39;&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">memorised</span><span class="p">))</span>
<span class="n">result</span><span class="p">[</span><span class="n">memorised</span><span class="p">]</span> <span class="o">=</span> <span class="mf">1.0</span>
<span class="k">return</span> <span class="n">result</span>
</code></pre></div>
<p>The <code>MemoizationPolicy</code> <em>recalls</em>, if it saw the current dialogue in the training data. If so, it returns what's in the training data with 100% confidence. That means if you stay in the "golden path" (the stories that were trained) the memoization policy does exactly what's in the training data.</p>
<h2>Who is pulling the strings?</h2>
<p>With the added debug output in the policy ensemble and the memoization policy, I saw that most of the time the memoization policy decided the next turn. In it self, this is not a bad thing. If a dialogue is exactly like in the training data, it is probably not a bad idea to do what the training data says.</p>
<p>However, when I disabled the memoization policy the neural network did not perform as expected.</p>
<div class="highlight"><pre><span></span><code><span class="n">Input</span><span class="o">:</span>
<span class="o">&gt;</span> <span class="n">hi</span>
<span class="n">Logging</span><span class="o">:</span>
<span class="mi">2018</span><span class="o">-</span><span class="mo">04</span><span class="o">-</span><span class="mi">26</span> <span class="mi">08</span><span class="o">:</span><span class="mi">54</span><span class="o">:</span><span class="mi">11</span> <span class="n">DEBUG</span> <span class="n">rasa_core</span><span class="p">.</span><span class="n">processor</span> <span class="o">-</span> <span class="n">Received</span> <span class="n">user</span> <span class="n">message</span> <span class="s">&#39;hi&#39;</span> <span class="n">with</span> <span class="n">intent</span> <span class="s">&#39;{&#39;</span><span class="n">name</span><span class="s">&#39;: &#39;</span><span class="n">greet</span><span class="s">&#39;, &#39;</span><span class="n">confidence</span><span class="s">&#39;: 0.9174638293066442}&#39;</span> <span class="kr">and</span> <span class="n">entities</span> <span class="s">&#39;[]&#39;</span>
<span class="mi">2018</span><span class="o">-</span><span class="mo">04</span><span class="o">-</span><span class="mi">26</span> <span class="mi">08</span><span class="o">:</span><span class="mi">54</span><span class="o">:</span><span class="mi">11</span> <span class="n">DEBUG</span> <span class="n">rasa_core</span><span class="p">.</span><span class="n">policies</span><span class="p">.</span><span class="n">ensemble</span> <span class="o">-</span> <span class="o">&lt;</span><span class="n">__main__</span><span class="p">.</span><span class="n">RestaurantPolicy</span> <span class="n">object</span> <span class="n">at</span> <span class="mh">0x12d3c6438</span><span class="o">&gt;</span> <span class="n">made</span> <span class="n">the</span> <span class="n">decision</span><span class="o">!</span>
<span class="mi">2018</span><span class="o">-</span><span class="mo">04</span><span class="o">-</span><span class="mi">26</span> <span class="mi">08</span><span class="o">:</span><span class="mi">54</span><span class="o">:</span><span class="mi">11</span> <span class="n">WARNING</span> <span class="n">rasa_core</span><span class="p">.</span><span class="n">processor</span> <span class="o">-</span> <span class="n">Circuit</span> <span class="n">breaker</span> <span class="n">tripped</span><span class="p">.</span> <span class="n">Stopped</span> <span class="n">predicting</span> <span class="n">more</span> <span class="n">actions</span> <span class="n">for</span> <span class="n">sender</span> <span class="s">&#39;default&#39;</span>
<span class="kr">Output</span><span class="o">:</span>
<span class="o">--&gt;</span> <span class="n">I</span><span class="s">&#39;m on it</span>
<span class="s">--&gt; I&#39;</span><span class="n">m</span> <span class="n">on</span> <span class="n">it</span>
<span class="o">--&gt;</span> <span class="n">I</span><span class="s">&#39;m on it</span>
<span class="s">--&gt; what kind of cuisine would you like?</span>
<span class="s">--&gt; I&#39;</span><span class="n">m</span> <span class="n">on</span> <span class="n">it</span>
<span class="o">--&gt;</span> <span class="n">I</span><span class="s">&#39;m on it</span>
<span class="s">--&gt; I&#39;</span><span class="n">m</span> <span class="n">on</span> <span class="n">it</span>
<span class="o">--&gt;</span> <span class="n">what</span> <span class="n">kind</span> <span class="kr">of</span> <span class="n">cuisine</span> <span class="n">would</span> <span class="n">you</span> <span class="n">like</span><span class="o">?</span>
<span class="o">--&gt;</span> <span class="n">I</span><span class="s">&#39;m on it</span>
<span class="s">--&gt; I&#39;</span><span class="n">m</span> <span class="n">on</span> <span class="n">it</span>
</code></pre></div>
<p>That's not looking good. To clarify: I removed some logging messages, but in principle the bot is producing more and more actions, more and more responses until the rasa core activates a "circuit breaker" so that the bot is not stuck in an endless loop. The rasa nlu classified the input correctly (intent <code>greet</code> and no entities mentioned) and the Restaurant policy made the decision.</p>
<p>Why is this happening? Short answer: <code>action_listen</code>. Long answer: there is a "<em>hidden</em>" action the bot can do: the listen action. After the user writes something the policy is activated again and again until it produces an <code>action_listen</code> where the bot awaits new input from the user. This way the bot is able to answer with more than one action. For example, if the user has requested to search for a restaurant, the bot can execute the action <code>action_on_it</code> - telling the user that it could take a while - and then it could execute the <code>bot.ActionSearchRestaurants</code> action.</p>
<p>That's a neat feature, but it seems to make problems with the neural network. The network (called the <code>KerasPolicy</code>) is asked what to do next. It predicts to say "<em>I'm on it</em>", sends the message to the user and informs the tracker. The tracker then takes the update dialogue and activates the policy again. Because there is something wrong with our neural net, it again predicts the <code>action_on_it</code> action and round and round it goes.</p>
<p>One other noteworthy case that seems to appear often is that after a users input the machine learning policy immediately predicts the <code>action_listen</code>. That way the bot simply is silent and awaits new inputs from the user.</p>
<h2>What's next?</h2>
<p>So the machine learning approach is not that robust and useful as it seemed! I don't want to say that it is not working, just that the parameters included in the tutorial are clearly not working out.</p>
<p>I will try to tweak some parameters and see if I can get the system to run in a more acceptable way. I will also look into the featurization of the dialogue state. Maybe there are some insight to why the current settings are not working.</p></content><category term="Dialogue"></category><category term="programming"></category><category term="rasa"></category><category term="chat-bot"></category><category term="dialogue systems"></category><category term="machine learning"></category></entry><entry><title>AstrologySort</title><link href="http://uhlo.github.io/astrologysort.html" rel="alternate"></link><published>2017-09-26T12:50:00+02:00</published><updated>2017-09-26T12:50:00+02:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2017-09-26:/astrologysort.html</id><summary type="html"><p>Some days ago I was bored and searched the internet for something to do. Something productive. Naturally, I went straight to the okayest subreddit I know (<a href="https://reddit.com/r/shittyprogramming">/r/shittyprogramming</a>) and searched for "idea" to get the hottest news on what one might productively do. And you will never guess what I've …</p></summary><content type="html"><p>Some days ago I was bored and searched the internet for something to do. Something productive. Naturally, I went straight to the okayest subreddit I know (<a href="https://reddit.com/r/shittyprogramming">/r/shittyprogramming</a>) and searched for "idea" to get the hottest news on what one might productively do. And you will never guess what I've found... *<em>clickbaiting intensifies</em>*</p>
<!-- more -->
<p>Well, maybe you've guessed it with the help of the big fat title above this post: <a href="https://www.reddit.com/r/shittyprogramming/comments/7168o5/idea_use_astrology_to_predict_branching_see_if_it/dn8mq9s/">someone suggested to use astrology for sorting</a>:</p>
<div class="reddit-embed" data-embed-media="www.redditmedia.com" data-embed-parent="false" data-embed-live="false" data-embed-uuid="a33077e8-f426-4688-8538-855043aa9a16" data-embed-created="2017-09-25T15:41:57.650Z"><a href="https://www.reddit.com/r/shittyprogramming/comments/7168o5/idea_use_astrology_to_predict_branching_see_if_it/dn8mq9s/">Comment</a> from discussion <a href="https://www.reddit.com/r/shittyprogramming/comments/7168o5/idea_use_astrology_to_predict_branching_see_if_it/">Idea: use astrology to predict branching, see if it works.</a>.</div>
<script async src="https://www.redditstatic.com/comment-embed.js"></script>
<p>While i liked the idea of using a long proven "<em>alternative source of truth</em>" (as I like to call it) for sorting, I didn't quite understand the predictions of this user concerning the runtime.</p>
<p>So I dusted off my vim and got to work.</p>
<h2>Some Research</h2>
<p>While thinking about how to realize an AstrologySort-algorithm, my first doubts in this project arose: <em>What is astrology? How could it ever be used to sort a list of numbers? How do I exit vim?</em></p>
<p>So, after I killed my terminal, I reflected on my scientific education and went straight to the most trustworthy source I know: Google. The complicated and precisely targeted search term "Astrology" revealed all the information needed: <a href="https://astrology.com">astrology.com</a>.</p>
<p><a href="https://astrology.com"><img alt="A screenshot of astrology.com" src="images/astrologycom_screenshot.png"></a></p>
<p>Almost instantly I spotted the "<em>Numerology</em>"-section of the website. I simply <em>knew</em> that this was the missing key to all my questions (except the one about how to exit vim).</p>
<p>But something still didn't feel quite right. Not scientific enough. What I needed was <em>hard evidence</em>, the kind that could be cited in papers and stuff. So I looked up "<em>Numerology</em>" in Wikipedia. And there it was: <a href="https://en.wikipedia.org/wiki/Numerology">A whole Wikipedia entry on this exact topic.</a> </p>
<blockquote>
<p>It has its own Wikipedia entry, therefore it must be real.</p>
</blockquote>
<p>With this, I was ready to end the research intensive part of this project.</p>
<h2>The Divine Source of Numbers</h2>
<p>The most well known tool of applied Numerology is of course the "<em>Daily Karmic Number</em>". I don't exactly know how it works, but I guess it has something to do with karma (so it's basically just like reddit). Every person has one Daily Karmic Number (DKN) for each day, but when I disabled cookies I got a new karmic number every time I loaded the page. Maybe I just have bad karma or something, but I went with it.</p>
<p><a href="https://www.astrology.com/us/games/game-daily-karmic-number.aspx"><img alt="Daily Karmic Number on astrology.com" src="images/astrologycom_dkn_screenshot.png"></a></p>
<p>So I postulate that if a number is your DKN it is better for you and it should therefore be at the front of a sorted list. Likewise, when comparing two numbers, it should be clear that the number that appears first as a DKN should be before the other.</p>
<div class="highlight"><pre><span></span><code><span class="n">the_list</span> <span class="o">=</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
<span class="n">sorted_list</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">while</span> <span class="n">the_list</span><span class="p">:</span>
<span class="n">dkn</span> <span class="o">=</span> <span class="n">get_dkn</span><span class="p">()</span>
<span class="k">if</span> <span class="n">dkn</span> <span class="ow">in</span> <span class="n">the_list</span><span class="p">:</span>
<span class="n">sorted_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">dkn</span><span class="p">)</span>
<span class="n">the_list</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">dkn</span><span class="p">)</span>
</code></pre></div>
<p>In the code example above, the <code>get_dkn</code>-function connects to astrology.com and fetches the latest DKN.</p>
<p>When I tested my code I got some very unusual and unexpected results: the numbers in the list weren't <em>sorted</em>. At least not by the definition I learned in algorithms 101.</p>
<blockquote>
<p>Sorted list: [9, 11, 3, 8, 1, 6, 3, 4]</p>
</blockquote>
<p>Turns out, the problem was not my implementation of the algorithm, but my expectations of it. The AstrologySort does not sort lists in a regular old-fashioned way. It sorts the numbers by <em>personal karmic ability</em>, which is much better than all the other sorting algorithms there are!</p>
<h2>The Performance Test</h2>
<p>But the word of that one redditor still remained in my head: "<em>It might even be slower than BogoSort.</em>"</p>
<p>That's why I ran some simple performance tests with BogoSort and AstrologySort. I tested with random-numbered (1 &lt;= x &lt;= 9) list that ranged in size from 1 to 10. I checked for the number of iterations it took and for the time.</p>
<iframe width="100%" height="400" frameborder="0" scrolling="no" src="//plot.ly/~uhlomuhlo/2.embed"></iframe>
<p>And as it turns out, the redditor was right! The chart shows clearly that while bogus sort is extremely fast for small lists, it is gets exponentially worse with larger lists. On the other hand, astrology sort is very slow for small lists and for large lists.</p>
<iframe width="100%" height="400" frameborder="0" scrolling="no" src="//plot.ly/~uhlomuhlo/1.embed"></iframe>
<p>The same with the number of steps to complete the sorting. While bogus sort shows its exponential behavior, astrology sort is more or less constant in the amount of steps it takes (if you look at it from far enough - and that's what astrology is all about).</p>
<p>So I just presented a constant time sorting algorithm, that sorts in such a special way that the numbers aren't even ordered afterwards! A miracle? No. It's just the way astrology works.</p></content><category term="Random"></category><category term="random"></category><category term="programming"></category><category term="highqualitybullshit"></category><category term="astrology"></category></entry><entry><title>Hello World 2.0!</title><link href="http://uhlo.github.io/hello-world-20.html" rel="alternate"></link><published>2017-08-22T10:53:00+02:00</published><updated>2017-08-22T10:53:00+02:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2017-08-22:/hello-world-20.html</id><summary type="html"><p>So this blog did not receive any love from me for quite some time. That's why I decided to revamp the whole thing and now it looks completely the same! Want to read me ramble about that topic for some paragraphs? Then read on...</p>
<!-- more -->
<h3>That one time I tried to …</h3></summary><content type="html"><p>So this blog did not receive any love from me for quite some time. That's why I decided to revamp the whole thing and now it looks completely the same! Want to read me ramble about that topic for some paragraphs? Then read on...</p>
<!-- more -->
<h3>That one time I tried to run a blog</h3>
<p>In the beginning, there was light. After that - in mid 2014 I think - I started this blog. And behold, I saw that all was well. I used <a href="http://octopress.org/">Octopress</a>, a Content Management Framework written in Ruby (Version 3 is promised to be coming since January 2015, so let's see how that will turn out). Octopress has nice tools for syntax highlighting and is basically a neat little static site generator extended by some blogging features.</p>
<p>But since Octopress is build in <em>Ruby</em>, a language that I have a love-hate relationship with, it soon came to problems. Ruby sucks (or at least it sucked back in the day) when it comes to managing packages. Back in 2013/2014 I did a lot of stuff with Ruby and Rails and every few weeks, some <em>gem</em> broke and I spend hours trying to fix that so I could continue to work.</p>
<p>The same thing happened with this blog. I could neither get Octopress to run, nor could I bring myself to debug it, mostly because I knew it would definitly break again.</p>
<p>Usually, the main apology for someone not blogging is the lack of time to write posts. But I was quite often willing to blog - I even set up a Wordpress blog. But the usual blogging systems are not right for me as they have all these features I don't use and are hard to customize.</p>
<h3>Riding the Pelican</h3>
<p>Now I took some time to set up a <a href="http://getpelican.com/">Pelican</a> blog. Which is basically the same thing, but with Python. It took some hours to get the old theme from Octopress running, but now everything (except comments and some minor other stuff) is working.</p>
<p>And I have to say, I'm quite happy about it. The only thing left is writing some blog posts. Let's see how this turns out...</p></content><category term="Meta"></category><category term="blog"></category><category term="meta"></category><category term="pelican"></category></entry><entry><title>U.N. Interstellar was her?</title><link href="http://uhlo.github.io/un-interstellar-was-her.html" rel="alternate"></link><published>2014-11-09T21:57:36+01:00</published><updated>2014-11-09T21:57:36+01:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2014-11-09:/un-interstellar-was-her.html</id><summary type="html"><p>So I've seen the new Christopher Nolan movie Interstellar yesterday! I had very mixed feelings about it and I will definitely see it again soon. The movie turned out pretty different that I had imagined and that threw me a bit off. This blog post however is not about the …</p></summary><content type="html"><p>So I've seen the new Christopher Nolan movie Interstellar yesterday! I had very mixed feelings about it and I will definitely see it again soon. The movie turned out pretty different that I had imagined and that threw me a bit off. This blog post however is not about the movie directly but about the music in it. So don't worry: no spoilers :)</p>
<!-- more -->
<h3>The Film Score</h3>
<p>The film score was written by Hans Zimmer. Apparently there isn't a reasonably successful movie that Hans Zimmer didn't make to music to. The score was a bit repetitive but that's okay because I liked the main theme. It reminded me a bit of the main theme of Nolan's previous movie Inception.
Anyways you can listen to an extended version of the main theme here:</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/wqnnRIwoxB8" frameborder="0" allowfullscreen></iframe>
<p><br/><br/></p>
<h3>I know that from somewhere...</h3>
<p>While watching the movie I had these feeling that up until a certain point in the progression of the melody I recognized it from somewhere but in the cinema I thought it was a <em>deja entendu</em>.</p>
<p>The next day I tried playing the melody on my e-piano and suddenly realize where my deja entendu came from: U.N. Owen was her? (or Flandre's Theme as many people call it).
Flandre's Theme is played during the extra stage boss fight in the vertical-scrolling game <a href="http://en.wikipedia.org/wiki/Touhou_Project">Touhou Project</a>.</p>
<p>I made a quick recording of my findings:</p>
<iframe width="100%" height="450" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/176120105&amp;auto_play=false&amp;hide_related=false&amp;show_comments=true&amp;show_user=true&amp;show_reposts=false&amp;visual=true"></iframe>
<p>I know, I know. It is barely similar. I had fun anyway :)</p></content><category term="Random"></category><category term="random"></category><category term="movies"></category><category term="music"></category></entry><entry><title>Compression Distance</title><link href="http://uhlo.github.io/compression-distance.html" rel="alternate"></link><published>2014-10-19T00:36:13+02:00</published><updated>2014-10-19T00:36:13+02:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2014-10-19:/compression-distance.html</id><summary type="html"><p>So I've recently looked into compression based distances as part of research for a masters course at the TU Berlin. What I found was a really interesting way of measuring distances between two textures (i.e. images).</p>
<!-- more -->
<h3>Distances? Images!?</h3>
<p>Yep. In principle one could measure some distances between any two …</p></summary><content type="html"><p>So I've recently looked into compression based distances as part of research for a masters course at the TU Berlin. What I found was a really interesting way of measuring distances between two textures (i.e. images).</p>
<!-- more -->
<h3>Distances? Images!?</h3>
<p>Yep. In principle one could measure some distances between any two objects. For example an acceptable distance measure between two images could be the difference in pixels or the difference in brightness. These distance measures all have their field of application but when <em>normal humans</em> talk about similarity in images they mean something different. Computers in general however have a hard time figuring out what we humans consider <em>similar</em>.</p>
<h3>Compression based distances</h3>
<p>One way to measure distances between objects in general is called compression distance. The way it works could be roughly described as</p>
<blockquote>
<p>The more efficiently an algorithm can compress object A given object B the more similar the two objects are.</p>
</blockquote>
<p>So what does that mean? As a little example: Take the sentence A: <em>"I like ice cream."</em> and compare it to the two sentences B: <em>"I like ice tea."</em> and C: <em>"Has Anyone Really Been Far Even as Decided to Use Even Go Want to do Look More Like?"</em>. Of course sentence A is more similar to B than C. But how could a computer measure that similarity? Easy! Just use a compression based distance.</p>
<p>When compressing <em>"I like ice cream."</em> together with <em>"I like ice tea."</em> a lot of information can be compressed (for example both sentences share the prefix <em>I like ice</em> in common). Compressing sentence A and C together would have a lot less potential for compression (none of the words in sentence A are contained in C).</p>
<p>And that's the concept of compression based distances</p>
<h3>Campana-Keogh-1</h3>
<p>No I didn't just sneezed. Campana-Keogh-1 or CK-1 for short is a neat little algorithm for measuring the distance between two images. And because no one wants to write complicated algorithms this one is very simple.</p>
<p>The idea is to let someone else do all the work: MPEG-1. MPEG-1 is used for compressing videos and it is really good at it! One very cool feature of such video encoding algorithms is the predictive frame. When using predictive frames only the difference to the preceding image is stored. So when two following images in a video are <em>similar</em> (which is often the case in movies), MPEG-1 only has to store the pixels that differ between the two images. Very neat, huh?</p>
<p>So what CK-1 is basically doing is to create two-frame videos. The smaller the resulting video-file the more similar the two images are. So the CK-1 algorithm basically just looks like this:</p>
<div class="highlight"><pre><span></span><code><span class="k">function</span><span class="w"> </span>distance<span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nf">CK1Distance</span><span class="p">(</span>x,y<span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="n">distance</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">mpegSize</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">y</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">mpegSize</span><span class="p">(</span><span class="n">y</span><span class="p">,</span><span class="n">x</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="n">distance</span><span class="w"> </span><span class="o">/</span><span class="p">=</span><span class="w"> </span><span class="n">mpegSize</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="n">x</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">mpegSize</span><span class="p">(</span><span class="n">y</span><span class="p">,</span><span class="n">y</span><span class="p">);</span><span class="w"></span>
<span class="w"> </span><span class="n">distance</span><span class="w"> </span><span class="s">−=</span><span class="w"> </span><span class="s">1</span><span class="p">;</span><span class="w"></span>
<span class="k">end</span><span class="w"></span>
</code></pre></div>
<p>The function <code>mpegSize(x,y)</code> just returns the size of an MPEG-1 video with two frames <code>x</code> and <code>y</code>. Simple as that. But the results are very nice!</p>
<h3>Who is this Mulder guy?</h3>
<p>So I've been watching some x-files episodes lately and because I always like to do stuff while I'm binging a TV series I decided to test the CK-1 algorithm on Mulder.</p>
<div style="width: 100%; text-algin: center;">
<img src="/images/whackamulder.png" alt="Some Screenshots of persons in an x-files episode"/>
</div>
<p>So I took 4 screenshots of Mulder and 2 from other people in similar poses. Then I ran these images through my self-written little CK-1 Java program (because who the heck has a license for Matlab??) and these were the results:</p>
<div class="highlight"><pre><span></span><code>d(mulder1.png,mulder1.png) = 0.0
d(mulder1.png,mulder2.png) = 0.6300178810907466
d(mulder1.png,mulder3.png) = 0.7965330333401515
d(mulder1.png,mulder4.png) = 0.9076955523269439
d(mulder1.png,someone.png) = 1.0699117676216914
d(mulder1.png,someone2.png) = 1.1179630519282
d(mulder2.png,mulder2.png) = 0.0
d(mulder2.png,mulder3.png) = 0.7866943866943867
d(mulder2.png,mulder4.png) = 0.882641168355454
d(mulder2.png,someone.png) = 1.0327691102945606
d(mulder2.png,someone2.png) = 1.080532305920907
d(mulder3.png,mulder3.png) = 0.0
d(mulder3.png,mulder4.png) = 0.8846048223210854
d(mulder3.png,someone.png) = 1.037555178268251
d(mulder3.png,someone2.png) = 0.9344867708807609
d(mulder4.png,mulder4.png) = 0.0
d(mulder4.png,someone.png) = 1.0254100592831557
d(mulder4.png,someone2.png) = 0.9044922962687953
d(someone.png,someone.png) = 0.0
d(someone.png,someone2.png) = 1.045421475903022
d(someone2.png,someone2.png) = 0.0
</code></pre></div>
<p>Okay, <code>mulder1</code> and <code>mulder2</code> are almost the same images. Some basic histogram analysis would have revealed that these two images would be the most <em>similar</em>. But the fact that <code>mulder3</code> is more similar to the other Mulders than to the random persons (even if the margin is not that high) is pretty neat. The <code>mulder4</code> images seems to be more similar to <code>someone2</code> than to <code>mulder1</code> but I think that's okay if we think about how simple and fast this algorithm is.</p>
<h3>Do some stuff!</h3>
<p>If you want to you can try my super cool CK-1 distance calculator on your own!
Just download it from <a href="/assets/CK1Java.jar">here</a> and run it like this:</p>
<div class="highlight"><pre><span></span><code>$ java -jar CK1Java.jar /path/to/folder/with/imgs/
- OR -
$ java -jar CK1Jar.jar img1.png img2.png
</code></pre></div>
<p>You just have to have <code>mencoder</code> installed and available before running.</p></content><category term="Programming"></category><category term="programming"></category><category term="movies"></category></entry><entry><title>My Octopress Blog</title><link href="http://uhlo.github.io/my-octopress-blog.html" rel="alternate"></link><published>2014-07-30T17:36:44+02:00</published><updated>2014-07-30T17:36:44+02:00</updated><author><name>Thilo Michael</name></author><id>tag:uhlo.github.io,2014-07-30:/my-octopress-blog.html</id><summary type="html"><p><strong><em>UPDATE: My blog now runs on pelican. Ignore this post. Really.</em></strong></p>
<p>So I have a new website/blog. It is made with the <a href="http://octopress.org">octopress</a> blogging framework. Octopress is rather different from all the content management systems and blogs I've seen before. Since version 2.0 it is based on <a href="https://github.com/jekyll/jekyll">jekyll …</a></p></summary><content type="html"><p><strong><em>UPDATE: My blog now runs on pelican. Ignore this post. Really.</em></strong></p>
<p>So I have a new website/blog. It is made with the <a href="http://octopress.org">octopress</a> blogging framework. Octopress is rather different from all the content management systems and blogs I've seen before. Since version 2.0 it is based on <a href="https://github.com/jekyll/jekyll">jekyll</a> and thus has a completetly different approach on content management and creation. It uses rake tasks to accomplish most of the work. Because octopress is a framework, the comfort for creating and managing content are rather sparse, but then again it is really powerfull if you know what you're doing.</p>
<!-- more -->
<h3>CMF instead of CMS</h3>
<p>Octopress is a content management framework, not a content management system. What that means in practice is that you would create a new blog entry with a rake task on your commandline like this:</p>
<div class="highlight"><pre><span></span><code>$ rake new_post<span class="o">[</span><span class="s2">&quot;My octopress blog&quot;</span><span class="o">]</span>
</code></pre></div>
<p>This command simply creates a markdown-file with some attributes predefined:</p>
<div class="highlight"><pre><span></span><code>---
layout: post
title: &quot;My octopress blog&quot;
date: 2014-07-30 17:36:44 +0200
comments: true
categories:
---
</code></pre></div>
<p>Below these attributes you can write your blog entry in markdown (with some cool octopress features included). If you want to <em>manage</em> your posts you can add an <code>author</code> or a <code>published</code> field. Because it's markdown you can easily inject some HTML into your post.</p>
<p>That's it. No user management, no unnecessary back-end menus. I really like it :)</p>
<h3>The power of simplicity</h3>
<p>Because octopress consists of a few HTML files (with templating) here and a few sass files there you can easily change the layout and the pages to match how you want them to be.</p>
<p>I really like the idea to use a real files to store the blog posts. I don't really need a fancy SQL database to store information about the user, about plugins and about my posts. Things like comments can be injected with <a href="https://disqus.com">disqus</a>. You can even write your post offline and then upload them as soon as you are back online. It just makes everything so much simpler.</p>
<p>The only downside to this is that you need to deploy your blog every time you post a new entry. But if you host your little blog on <a href="http://heroku.com">Heroku</a> that is less of a problem.
<strong><em>Update: This blog is no longer running on Heroku</em></strong></p>
<h3>Not for everyone</h3>
<p>Octopress however is not suited for everyone. Multi-author blogs are supported, but writing my name into every post does not do it for most people. Implementing your own comment system is nearly impossible without setting up a database.</p>
<p>But octopress does not want to be for everyone. It describes itself as "a blogging framework for hackers". And that is exactly what it is. For people who just want to run their little blog, share their thoughts and ideas and also want to some really cool syntax highlight ;) its a really impressive alternative.</p></content><category term="Meta"></category><category term="blog"></category><category term="meta"></category><category term="octopress"></category></entry></feed>