Skip to content

Commit

Permalink
Use connect options for gutenberg, resolves #570
Browse files Browse the repository at this point in the history
  • Loading branch information
seanmor5 committed May 14, 2024
1 parent 0d5561b commit 88c7823
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
4 changes: 3 additions & 1 deletion examples/generative/text_generator.exs
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,10 @@ defmodule TextGenerator do
end

def run do
options = [transport_opts: [signature_algs_cert: :ssl.signature_algs(:default, :"tlsv1.3") ++ [sha: :rsa]]]

normalized_book_text =
Req.get!(@download_url).body
Req.get!(@download_url, connect_options: options).body
|> String.downcase()
|> String.replace(~r/[^a-z \.\n]/, "")
|> String.to_charlist()
Expand Down
3 changes: 2 additions & 1 deletion notebooks/text/lstm_generation.livemd
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ The one that we will use for this experiment is [Alice's Adventures in Wonderlan
```elixir
# Change the URL if you'd like to experiment with other books
download_url = "https://www.gutenberg.org/files/11/11-0.txt"
options = [transport_opts: [signature_algs_cert: :ssl.signature_algs(:default, :"tlsv1.3") ++ [sha: :rsa]]]

book_text = Req.get!(download_url).body
book_text = Req.get!(download_url, connect_options: options).body
```

First of all, we need to normalize the content of the book. We are only interested in the sequence of English characters, periods and new lines. Also currently we don't care about the capitalization and things like apostrophe so we can remove all other unknown characters and downcase everything. We can use a regular expression for that.
Expand Down

0 comments on commit 88c7823

Please sign in to comment.