Karpathy char-rnn.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "start_time": "2019-08-19T12:39:40.483Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data has 4573338 characters, 67 unique.\n",
      "iter 0, loss: 105.117317\n",
      "iter 10000, loss: 72.108401\n",
      "iter 20000, loss: 61.756417\n",
      "iter 30000, loss: 58.299910\n",
      "iter 40000, loss: 57.068671\n",
      "iter 50000, loss: 55.809686\n",
      "iter 60000, loss: 55.884134\n",
      "iter 70000, loss: 54.308297\n",
      "*************** shows thou hast been in l &&&&&&&&&&&&&&&\n",
      "----\n",
      " te doue thules Von\n",
      "What hos harch of, har go.\n",
      "\n",
      "TRUCER:\n",
      "You lery yourher stove lomay, with a outhiouf \n",
      "----\n",
      "iter 80000, loss: 54.430582\n",
      "***************  looks are sad, your chee &&&&&&&&&&&&&&&\n",
      "----\n",
      " fard ford:\n",
      "She aur.\n",
      "\n",
      "RIORUS:\n",
      "Whithellese; herouk for andishage now hor hettrie theres ind to chid do \n",
      "----\n",
      "iter 90000, loss: 54.194205\n",
      "*************** nt;\n",
      "Of heaven, not me, ma &&&&&&&&&&&&&&&\n",
      "----\n",
      " erighensollow ute.\n",
      "\n",
      "Sorded and tad aisen sour be sor that pralldtend hived at uppeose, rroyse's coun \n",
      "----\n",
      "iter 100000, loss: 54.347676\n",
      "*************** ; or the men of Troy\n",
      "Are  &&&&&&&&&&&&&&&\n",
      "----\n",
      "  fortollir.\n",
      "\n",
      "AROVLE:\n",
      "Eou.\n",
      "\n",
      "PAROFKENY ENYOT:\n",
      "Aaby 'tiint.\n",
      "\n",
      "BARNIUPDUFh, anveruthor Cthat is yes, thit \n",
      "----\n",
      "iter 110000, loss: 53.757549\n",
      "*************** pon the sightless courier &&&&&&&&&&&&&&&\n",
      "----\n",
      " ronst, jore,\n",
      "Tuk's fattes hear the liy,\n",
      "And to heir ghing as to a ob hall\n",
      "Seroped to matung!\n",
      "\n",
      "nzange \n",
      "----\n",
      "iter 120000, loss: 54.255291\n",
      "*************** k,\n",
      "And would afford my sp &&&&&&&&&&&&&&&\n",
      "----\n",
      " ee. ATNIAS:\n",
      "Makere,''s be! monto, tlead Sand ames come brar; the toor nas to fat most, oftris by\n",
      "y m \n",
      "----\n",
      "iter 130000, loss: 54.035828\n",
      "*************** ood upon.\n",
      "\n",
      "MORTIMER:\n",
      "I un &&&&&&&&&&&&&&&\n",
      "----\n",
      " wsmonce oto Pand redein but livioss evas thoware reathoen Beever any thoaldiis!\n",
      "\n",
      "KINA:\n",
      "Bbounowle dra \n",
      "----\n",
      "iter 140000, loss: 53.610863\n",
      "***************  PAGE:\n",
      "What, have I scape &&&&&&&&&&&&&&&\n",
      "----\n",
      " ferthar, alame an mO:\n",
      "Wiy coMYull vave tha paves vick. that plued ingut dithe loughs and anm if e bl \n",
      "----\n",
      "iter 150000, loss: 53.539463\n",
      "*************** nd the dull\n",
      "elements of e &&&&&&&&&&&&&&&\n",
      "----\n",
      "  for y, suman ence,\n",
      "Th hactuen, unool my if, as Cour pouso; be to con dolaing\n",
      "Kes be ee the funsteks \n",
      "----\n",
      "iter 160000, loss: 53.419012\n",
      "*************** uch wit thou lackest.\n",
      "\n",
      "AP &&&&&&&&&&&&&&&\n",
      "----\n",
      " t you: the ouf and thes bond, gract the sang y notigrent I weng for be ut:\n",
      "I and in mush whe:\n",
      "Your o \n",
      "----\n",
      "iter 170000, loss: 53.270409\n",
      "*************** is this cloak of?\n",
      "\n",
      "SIMPCO &&&&&&&&&&&&&&&\n",
      "----\n",
      " nt, no-do cood and of him:\n",
      "You?\n",
      "Ayent: carind ler, as ard com?\n",
      "\n",
      "SISTEAST:\n",
      "That havike hid ghear toum \n",
      "----\n",
      "iter 180000, loss: 52.903339\n",
      "***************  swart, prodigious,\n",
      "Patch &&&&&&&&&&&&&&&\n",
      "----\n",
      " mighfingsiin as as his beabdy at of and myswad beld halcOO:\n",
      "I and toured so and fur ommed:\n",
      "Wher of t \n",
      "----\n",
      "iter 190000, loss: 52.028614\n",
      "*************** gue upon you all! His roy &&&&&&&&&&&&&&&\n",
      "----\n",
      " u mamark s mensuro'vizes?\n",
      "\n",
      "MY\n",
      "Whourt lale:\n",
      "e and himme, falm thall, ust to no bedbonks, appade if ki \n",
      "----\n",
      "iter 200000, loss: 52.144207\n",
      "*************** wn?\n",
      "Have we more sons? or &&&&&&&&&&&&&&&\n",
      "----\n",
      " e.\n",
      "\n",
      "MARY.\n",
      "And ser leer ant chere laysorew thill op:\n",
      "Thet stung\n",
      "Lath My of amming thack\n",
      "The with\n",
      "Whos \n",
      "----\n",
      "iter 210000, loss: 51.793904\n",
      "*************** , Warwick, thou art worth &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "Howns youredeston head to dois I mayfice folde withelocred,\n",
      "What he my; whenter here pigg\n",
      "Hy leshar \n",
      "----\n",
      "iter 220000, loss: 51.844944\n",
      "*************** er is the friend of men.\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " ntek hor tonie, that prom abehan comes stle, some swinne.\n",
      "\n",
      "I PAMYOFAENIAEN:\n",
      "It gaje mind. I so R'K,  \n",
      "----\n",
      "iter 230000, loss: 52.026544\n",
      "*************** thank God\n",
      "and my cold blo &&&&&&&&&&&&&&&\n",
      "----\n",
      " her?\n",
      "\n",
      "PRINIA:\n",
      "Whill to of that dood the fist has and win,\n",
      "so Laklored abrainent good hitht theircise \n",
      "----\n",
      "iter 240000, loss: 51.510241\n",
      "*************** ight.\n",
      "\n",
      "KENT:\n",
      "That, from y &&&&&&&&&&&&&&&\n",
      "----\n",
      " '\n",
      "Thim weldich Lextoon thou but seaves,\n",
      "I op the him sar thilow not, oult tho'tires stakl?\n",
      "\n",
      "SEOThave \n",
      "----\n",
      "iter 250000, loss: 51.694970\n",
      "*************** p your word, Silvius, tha &&&&&&&&&&&&&&&\n",
      "----\n",
      "  to-wearecg this So a mones it ofs cridld I amnone ppisio, crairx on ebeather him I was thous\n",
      "oubdte \n",
      "----\n",
      "iter 260000, loss: 51.804117\n",
      "*************** elodious bird, it sung\n",
      "Sw &&&&&&&&&&&&&&&\n",
      "----\n",
      "  rase of dive Stooc sle in thit a'd to have thangod, jeilemare is in whe louch arsw Prach\n",
      "meds\n",
      "Pair' \n",
      "----\n",
      "iter 270000, loss: 51.962618\n",
      "*************** , if I were disposed to s &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "Ruckamill bivinst to and with in a menon weng us med!\n",
      "\n",
      "TUZRLIUS:\n",
      "Ostus's t me bur disoise,\n",
      "Catily.\n",
      " \n",
      "----\n",
      "iter 280000, loss: 51.592427\n",
      "*************** Y:\n",
      "I forgive him.\n",
      "\n",
      "SUFFOL &&&&&&&&&&&&&&&\n",
      "----\n",
      "  ICERNAN:\n",
      "The in?\n",
      "\n",
      "KESANE:\n",
      "Hast that and aust noul ataith thou have ay sice semot, gothour this this \n",
      "----\n",
      "iter 290000, loss: 52.250694\n",
      "***************  so may you miss me;\n",
      "But  &&&&&&&&&&&&&&&\n",
      "----\n",
      " crisinysirnsninct.\n",
      "Ie live 'pone dace\n",
      "It but not for, his\n",
      "Dedus,\n",
      "Thim I aspe, caunt\n",
      "Theibla sut his  \n",
      "----\n",
      "iter 300000, loss: 51.990726\n",
      "*************** devised\n",
      "courtesy. I think &&&&&&&&&&&&&&&\n",
      "----\n",
      " es his nom wats is so nead beer blanj; igh ass slowemre sin.\n",
      "Whiler Be pady,\n",
      "I must arton tenang in  \n",
      "----\n",
      "iter 310000, loss: 51.378835\n",
      "*************** tain of the issue any way &&&&&&&&&&&&&&&\n",
      "----\n",
      " her hore so fraads: Criss, I knownis, will vills, my oftat the you she sere Cains:\n",
      "Age roved'ck in.\n",
      " \n",
      "----\n",
      "iter 320000, loss: 51.431351\n",
      "*************** ly to be great\n",
      "Is not to  &&&&&&&&&&&&&&&\n",
      "----\n",
      " ithif.\n",
      "\n",
      "PESTRUCECRY:\n",
      "Fer'd and of the haal seratfe 'll ant by mn his aie to, lilen; Sothere sichis,\n",
      " \n",
      "----\n",
      "iter 330000, loss: 51.083300\n",
      "***************  say nay to that.\n",
      "\n",
      "ANTIPH &&&&&&&&&&&&&&&\n",
      "----\n",
      " ht dlyes\n",
      "ser ar-sher my threed that wht my het wothe of bit venwiss, is coult be amd me masthose we  \n",
      "----\n",
      "iter 340000, loss: 51.977557\n",
      "*************** the very extremest inch o &&&&&&&&&&&&&&&\n",
      "----\n",
      "  threofde, the, be, stoon wrome; the bray, gouce and muster, pears.\n",
      "\n",
      "USREN:\n",
      "Yen she?\n",
      "\n",
      "PADTOVH:\n",
      "I of  \n",
      "----\n",
      "iter 350000, loss: 51.635742\n",
      "*************** ort, gone; my queen\n",
      "Upon  &&&&&&&&&&&&&&&\n",
      "----\n",
      " , as:\n",
      "\n",
      "Se prunger:\n",
      "II hink;\n",
      "Bealdiond Imarses Sol sthen\n",
      "The peread'd\n",
      "Wno proud kn ble. be busd sake  \n",
      "----\n",
      "iter 360000, loss: 51.583573\n",
      "*************** f the knight what\n",
      "my offe &&&&&&&&&&&&&&&\n",
      "----\n",
      " then fith woradrumiosty.\n",
      "\n",
      "FFALVI:\n",
      "I noty,\n",
      "Sabe out by Doonce nom ra$thish fet of lith and rastoring  \n",
      "----\n",
      "iter 370000, loss: 50.682282\n",
      "*************** y\n",
      "business, and I will me &&&&&&&&&&&&&&&\n",
      "----\n",
      "  what!\n",
      "\n",
      "\n",
      "ILIUS I ssi!\n",
      "'s my mave heninifice, osh o, nied the mages to Ladst hophtld I wall he brever \n",
      "----\n",
      "iter 380000, loss: 51.085364\n",
      "*************** ICHARD II:\n",
      "What says he?\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " f dhothath sparde um ble tave to wargius--vor, my dnows\n",
      "Frridst you Anged't theeprow,\n",
      "What crokes,\n",
      "a \n",
      "----\n",
      "iter 390000, loss: 51.063271\n",
      "*************** , here's for my father's  &&&&&&&&&&&&&&&\n",
      "----\n",
      "  id an now the hill fries be the hast manme for hade my this durkill one loan.\n",
      "\n",
      "SKING RING:\n",
      "Whine ve \n",
      "----\n",
      "iter 400000, loss: 50.789428\n",
      "*************** erty, I am\n",
      "custom-shrunk. &&&&&&&&&&&&&&&\n",
      "----\n",
      "  dake me and like frist lavise,\n",
      "If hel whas Wall besers shallimed staden is tarrerss is lof tole.\n",
      "\n",
      "G \n",
      "----\n",
      "iter 410000, loss: 50.603261\n",
      "*************** myself am Naples,\n",
      "Who wit &&&&&&&&&&&&&&&\n",
      "----\n",
      " , too wehe-d gake\n",
      "Mareat them comon fiin thus reet im thour I praye, stromes,\n",
      "To thou wa Come knaw\n",
      "d \n",
      "----\n",
      "iter 420000, loss: 50.136038\n",
      "*************** NWALL:\n",
      "Shut up your doors &&&&&&&&&&&&&&&\n",
      "----\n",
      " on, a blot?\n",
      "\n",
      "EED: inturs the joopat\n",
      "Zore Heavry cang then gakntre fure\n",
      "And no sisen spourd meathe le \n",
      "----\n",
      "iter 430000, loss: 51.211468\n",
      "*************** pillow:\n",
      "But if thy love w &&&&&&&&&&&&&&&\n",
      "----\n",
      " uth mued yis, wolllyeno, micese the bmeat noto-tray hormagoet\n",
      "Go\n",
      "Andar\n",
      "It a good quer the noumaibats \n",
      "----\n",
      "iter 440000, loss: 50.631999\n",
      "*************** e god\n",
      "Priapus, and undo a &&&&&&&&&&&&&&&\n",
      "----\n",
      " d deer winds to sof is\n",
      "goldis jear dir\n",
      "I's, spevere the whose, think me a corthgite wead hes shert,\n",
      " \n",
      "----\n",
      "iter 450000, loss: 51.387089\n",
      "*************** passion and of lenity,\n",
      "To &&&&&&&&&&&&&&&\n",
      "----\n",
      " raks 'tcus tted,\n",
      "My coursons shalt no doncious I'lloef, coth,\n",
      "Bnterbed, thouth pleathis vilt you is  \n",
      "----\n",
      "iter 460000, loss: 50.367063\n",
      "*************** r,\n",
      "One Gilbert Peck, his  &&&&&&&&&&&&&&&\n",
      "----\n",
      " ath, buter the eo wot in thecelad,\n",
      "\n",
      "MAROUTEN:\n",
      "Whisly; iff, prewesteke allyt cousse and me heal I tha \n",
      "----\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter 470000, loss: 51.371036\n",
      "*************** TES:\n",
      "He'll tickle it for  &&&&&&&&&&&&&&&\n",
      "----\n",
      " otediestuaque: lord; thum mat, fs in ine,, even! Gom.\n",
      "\n",
      "CALINI:\n",
      "While ndaimes in, glernectand dablie  \n",
      "----\n",
      "iter 480000, loss: 51.317409\n",
      "***************  be the lady: thou shalt  &&&&&&&&&&&&&&&\n",
      "----\n",
      " had, and rownd,\n",
      "A doe ving it nom he pir to herrant\n",
      "For here tlent wither the reestiandees; heck the \n",
      "----\n",
      "iter 490000, loss: 51.195915\n",
      "*************** t this,--\n",
      "Away at once wi &&&&&&&&&&&&&&&\n",
      "----\n",
      " h deatien blessonsway stall faperriend the with contlf vethell thim? And loke. You shall aphere he P \n",
      "----\n",
      "iter 500000, loss: 50.995043\n",
      "*************** s more into command\n",
      "Than  &&&&&&&&&&&&&&&\n",
      "----\n",
      " o this the lipos, Hispliek an my is faves\n",
      "SHOMNENDZLLY:\n",
      "Thou 'gale, rust sures bustiod\n",
      "Thour ther me \n",
      "----\n",
      "iter 510000, loss: 51.083051\n",
      "*************** d sleep within mine inn,\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " our!;\n",
      "Mys let; to whe, Baste, and the wasich he earder e teee a onkeaud of end lefthathes npentere;  \n",
      "----\n",
      "iter 520000, loss: 51.211366\n",
      "*************** t eat thy dead vomit up,\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " hee.\n",
      "\n",
      "KING Hight in\n",
      "it so not on am you wife onad as in gesthe it with my like ghis\n",
      "zing of proobed  \n",
      "----\n",
      "iter 530000, loss: 50.855902\n",
      "*************** \n",
      "And I am something curio &&&&&&&&&&&&&&&\n",
      "----\n",
      " lathave\n",
      "Talred\n",
      "Thip have my makeet you caster--BARATHOLES:\n",
      "Scoytundee this shall me mess'd utcuns is \n",
      "----\n",
      "iter 540000, loss: 50.754285\n",
      "*************** \n",
      "Perchance he is not drow &&&&&&&&&&&&&&&\n",
      "----\n",
      " O ferave for not\n",
      "Thon mester a thd aF all monourthferuse.\n",
      "\n",
      "ERMILAR:\n",
      "I sirt: I spillint in takale mor \n",
      "----\n",
      "iter 550000, loss: 50.645429\n",
      "*************** ARTIUS:\n",
      "O general,\n",
      "Here i &&&&&&&&&&&&&&&\n",
      "----\n",
      " RCTER:\n",
      "Whor\n",
      "Ond thilewarnire brakused rnkitin sors are, thee touses qefented\n",
      "The prirs? Ving cad wor \n",
      "----\n",
      "iter 560000, loss: 49.862074\n",
      "*************** nd brothers?\n",
      "Tell me, tho &&&&&&&&&&&&&&&\n",
      "----\n",
      " e now\n",
      "As loblds; to blord, to sime unclour lich shoul, the genonce, him think.\n",
      "\n",
      "OASY VY:\n",
      "Cait sheer  \n",
      "----\n",
      "iter 570000, loss: 50.688316\n",
      "*************** than will to go:\n",
      "Come, de &&&&&&&&&&&&&&&\n",
      "----\n",
      " 's sace, Her of wer lell's wooly, my canimple, to bardisce\n",
      "Dureatsr hen, lives:\n",
      "Minst when is him th \n",
      "----\n",
      "iter 580000, loss: 50.147759\n",
      "*************** \n",
      "Clown:\n",
      "How now! canst st &&&&&&&&&&&&&&&\n",
      "----\n",
      " Kentighter wexcht!\n",
      "Bease cinter:\n",
      "Now I at lawy rite if hen thy pring hivise me marturf,\n",
      "Your the goo \n",
      "----\n",
      "iter 590000, loss: 50.176646\n",
      "*************** ted to your worship,\n",
      "I am &&&&&&&&&&&&&&&\n",
      "----\n",
      "  gamet nce on to ancusied blearclat't.\n",
      "\n",
      "CHHLEOS:\n",
      "Waysillee us no wif o and aty Tip itsint;\n",
      "Thene on  \n",
      "----\n",
      "iter 600000, loss: 49.779400\n",
      "*************** PEDRO:\n",
      "Come, let us hence &&&&&&&&&&&&&&&\n",
      "----\n",
      "  ame. Breaired an there,\n",
      "ULO:\n",
      "Mance d hogh't othe, whice, am melles I stay'd thencaisiunther sturt,\n",
      " \n",
      "----\n",
      "iter 610000, loss: 50.729170\n",
      "*************** SCARUS:\n",
      "I had a wound her &&&&&&&&&&&&&&&\n",
      "----\n",
      " e\n",
      "simed sead he lave wrougitingalld bustire\n",
      "with worvither our Mads.\n",
      "\n",
      "COMPEBARD:\n",
      "Goed with Ancesent  \n",
      "----\n",
      "iter 620000, loss: 49.463695\n",
      "*************** t a gentleman and well de &&&&&&&&&&&&&&&\n",
      "----\n",
      " oust yet, seareld lime extroor 'TAMAND:\n",
      "Ret.\n",
      "\n",
      "ROPELI, he\n",
      "jongld\n",
      "paply, and ofs nocer a live die that \n",
      "----\n",
      "iter 630000, loss: 50.619517\n",
      "*************** r:\n",
      "Unless my study and my &&&&&&&&&&&&&&&\n",
      "----\n",
      " ice.\n",
      "Nall to grewort wath a\n",
      "this ank Reitht to loves best, mor brous, oncet ip tiklyen,\n",
      "Thing this l \n",
      "----\n",
      "iter 640000, loss: 49.729437\n",
      "*************** re I'll entreat you\n",
      "Writt &&&&&&&&&&&&&&&\n",
      "----\n",
      " y; axtaimol this to?\n",
      "I losifly wife?:\n",
      "Congnish acthe, ay on As mindence, the my fiath of'll ind:\n",
      "For \n",
      "----\n",
      "iter 650000, loss: 50.761044\n",
      "*************** o him! Jupiter forbid,\n",
      "An &&&&&&&&&&&&&&&\n",
      "----\n",
      " ulA:\n",
      "He with of in with thou do hall I'll a kinge the so speesurm thoue.\n",
      "\n",
      "PARON:\n",
      "Anain tlates toobed \n",
      "----\n",
      "iter 660000, loss: 50.483477\n",
      "*************** old\n",
      "That is not often vou &&&&&&&&&&&&&&&\n",
      "----\n",
      " nce sle ta.\n",
      "\n",
      "ANTALE:\n",
      "What shand that high, shope pronmenight stase Teatrnd be seno marnert, ie but w \n",
      "----\n",
      "iter 670000, loss: 50.920084\n",
      "*************** When roasted crabs hiss i &&&&&&&&&&&&&&&\n",
      "----\n",
      " ANDONA:\n",
      "Naid.\n",
      "\n",
      "PONDIN:\n",
      "So shact commay, ank.\n",
      "welf-neep in he I reagllan,\n",
      "Horh to.\n",
      "\n",
      "CORIRNAN:\n",
      "Bony th \n",
      "----\n",
      "iter 680000, loss: 50.928422\n",
      "*************** I fear we shall stay too  &&&&&&&&&&&&&&&\n",
      "----\n",
      "  sounters,--'d yet then, wite did Sincerse a me firt\n",
      "Colius, no.\n",
      "Tillle, slate,\n",
      "And she dendessurono \n",
      "----\n",
      "iter 690000, loss: 50.394156\n",
      "***************  English.\n",
      "\n",
      "DOCTOR CAIUS:\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " beficken, me a cance gaxt's sere. I of may you, he poome\n",
      "at of you, his saymant-\n",
      "Yed. Wheredy.\n",
      "\n",
      "PARE \n",
      "----\n",
      "iter 700000, loss: 50.386662\n",
      "*************** se of mortality.\n",
      "Let me s &&&&&&&&&&&&&&&\n",
      "----\n",
      " ay, go ce sughought her shaikir, cnighint: It the guent I ball is onould knew\n",
      "That that juthound\n",
      "par \n",
      "----\n",
      "iter 710000, loss: 50.503181\n",
      "*************** hbourhood,\n",
      "Instruction, m &&&&&&&&&&&&&&&\n",
      "----\n",
      " eres\n",
      "My wis hore so in your ebit if go-do door my a, gexodvenst with\n",
      "noves the pount..\n",
      "\n",
      "HAMLPHABOBHA \n",
      "----\n",
      "iter 720000, loss: 50.305522\n",
      "*************** ns our doom.\n",
      "\n",
      "Post:\n",
      "Great &&&&&&&&&&&&&&&\n",
      "----\n",
      "  thou all all roon?\n",
      "\n",
      "CHIMIUC,\n",
      "That ame and my stoWho word tives noteatisest O mate Lear Jond the man \n",
      "----\n",
      "iter 730000, loss: 50.157364\n",
      "*************** coal;\n",
      "The breath of heave &&&&&&&&&&&&&&&\n",
      "----\n",
      " t Cwuit, is cursent my her fums, in and the our woy, maschfloce?\n",
      "\n",
      "Saques mone! chis cands\n",
      "Tiads inge \n",
      "----\n",
      "iter 740000, loss: 49.212370\n",
      "*************** ed, of all.\n",
      "I do not know &&&&&&&&&&&&&&&\n",
      "----\n",
      " re th.\n",
      "\n",
      "HAMOLAQUF:\n",
      "How here uctuble to dent coure him! and thou, the joult:\n",
      "Abou queaty; harnontuy!\n",
      " \n",
      "----\n",
      "iter 750000, loss: 49.680435\n",
      "***************  cousin\n",
      "Tybalt, Lucio and &&&&&&&&&&&&&&&\n",
      "----\n",
      " int,\n",
      "Do this same\n",
      "Thraters to spef afle hercht fich aten thee what their,\n",
      "Whe some If bet'd my lows  \n",
      "----\n",
      "iter 760000, loss: 49.238383\n",
      "*************** he raven rook'd her on th &&&&&&&&&&&&&&&\n",
      "----\n",
      " e haven of Tilk,\n",
      "And grow and comede him of offered theard;\n",
      "For for my lowy e:\n",
      "The thead chare themb \n",
      "----\n",
      "iter 770000, loss: 49.350951\n",
      "***************  will know his purpose.\n",
      "W &&&&&&&&&&&&&&&\n",
      "----\n",
      " lestion; by they:\n",
      "Asedi'd\n",
      "And.\n",
      "\n",
      "Py,\n",
      "Ine we dout iple: kef Claed to qualy, himes the the panks!\n",
      "\n",
      "QUGR \n",
      "----\n",
      "iter 780000, loss: 49.732569\n",
      "*************** ur brother; spare not to  &&&&&&&&&&&&&&&\n",
      "----\n",
      " pica ter s chtit to be hearence. Frondesce, etteep.\n",
      "\n",
      "FFRETO:\n",
      "I'll mo, rbe pall poroule firs as by co \n",
      "----\n",
      "iter 790000, loss: 49.995470\n",
      "***************  spoken well.\n",
      "Were we bef &&&&&&&&&&&&&&&\n",
      "----\n",
      " boon the gake kneathe, wain faike thou that thenst carns eardew the on sasend I diblippos kale yen,  \n",
      "----\n",
      "iter 800000, loss: 49.564643\n",
      "***************  so much pains?\n",
      "\n",
      "VALENTIN &&&&&&&&&&&&&&&\n",
      "----\n",
      " SAgested\n",
      "a a kian\n",
      "more fope mortud of theavend as will virs, me just chares\n",
      "Theod,\n",
      "Tood he kingearri \n",
      "----\n",
      "iter 810000, loss: 49.761146\n",
      "*************** hen I bid.\n",
      "Of my word, I  &&&&&&&&&&&&&&&\n",
      "----\n",
      " ave facy us you, on be we bar,\n",
      "Buth of my 't,\n",
      "Recolk mourter wome beland make fith seer mod, a recar \n",
      "----\n",
      "iter 820000, loss: 49.549591\n",
      "*************** w I change my mind,\n",
      "And p &&&&&&&&&&&&&&&\n",
      "----\n",
      " ench wat; Corticustord over they haste, to fiy.\n",
      "\n",
      "Awty? Come? I amblase I fols!\n",
      "I neverd Came I as go \n",
      "----\n",
      "iter 830000, loss: 49.605202\n",
      "*************** t, then his laws, in fill &&&&&&&&&&&&&&&\n",
      "----\n",
      " aist: garth offist durs:\n",
      "No!\n",
      "\n",
      "SALUS:\n",
      "That LO.\n",
      "Leved.\n",
      "\n",
      "Came to raknstnices wom:\n",
      "Anks moresseaber: for \n",
      "----\n",
      "iter 840000, loss: 50.254388\n",
      "*************** y,\n",
      "Wherein doth sit the d &&&&&&&&&&&&&&&\n",
      "----\n",
      "  rlk,\n",
      "Mast my goll,\n",
      "And tha watthirnse fect boouth, aron make thee:\n",
      "Go mugkind rel resy an?\n",
      "Ugun\n",
      "Hon \n",
      "----\n",
      "iter 850000, loss: 50.461236\n",
      "*************** hus,\n",
      "expecting thy reply, &&&&&&&&&&&&&&&\n",
      "----\n",
      " e'De conged,\n",
      "And we and brakes, thy pebpantel:\n",
      "Thrive:\n",
      "Your putung for eat,\n",
      "Why, thou this, hape for \n",
      "----\n",
      "iter 860000, loss: 49.952614\n",
      "*************** u keep!\n",
      "\n",
      "FALSTAFF:\n",
      "Where' &&&&&&&&&&&&&&&\n",
      "----\n",
      " te gain be and kir woy?\n",
      "\n",
      "KINGO:\n",
      "O, I any brover me.\n",
      "\n",
      "ONRUS;\n",
      "Sailes from bangair:\n",
      "Sur a we his and da \n",
      "----\n",
      "iter 870000, loss: 49.466400\n",
      "*************** til my eyelids will no lo &&&&&&&&&&&&&&&\n",
      "----\n",
      "  a gaifat tave it your and buirustine says not I's stidk hase since; out Hermolds,--\n",
      "beaner thur, is \n",
      "----\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter 880000, loss: 49.432145\n",
      "***************  death and treachery.\n",
      "\n",
      "KI &&&&&&&&&&&&&&&\n",
      "----\n",
      " have sough fose;\n",
      "In frow good thou gakenh the give hoirn, sumponodted\n",
      "That I,\n",
      "Whth to the his hears  \n",
      "----\n",
      "iter 890000, loss: 50.014821\n",
      "***************  state\n",
      "What I have done t &&&&&&&&&&&&&&&\n",
      "----\n",
      " me well peludinss the mandius.\n",
      "\n",
      "DASTNRIMUS:\n",
      "Ine he by of am beherescifund; ro\n",
      "made\n",
      "But weshor.\n",
      "\n",
      "VAST \n",
      "----\n",
      "iter 900000, loss: 50.075048\n",
      "*************** dogs,\n",
      "Of no esteem: I, dr &&&&&&&&&&&&&&&\n",
      "----\n",
      " ed fatherp\n",
      "Comn my to isot.\n",
      "If tomm though it thend of hore strongiges.\n",
      "\n",
      "OGHIRAS:\n",
      "Aveto the benghf g \n",
      "----\n",
      "iter 910000, loss: 49.603100\n",
      "***************  be imprison'd,\n",
      "Kept in a &&&&&&&&&&&&&&&\n",
      "----\n",
      " of doth\n",
      "mey store, axantunele.\n",
      "\n",
      "ANEBUS:\n",
      "Nor, sithen; well'll nat that be ainty heres ass picers, how \n",
      "----\n",
      "iter 920000, loss: 48.990331\n",
      "*************** US:\n",
      "Away!\n",
      "\n",
      "MENENIUS:\n",
      "How! &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "\n",
      "VIR:\n",
      "Lear thener would\n",
      "Wore that it lork presfle\n",
      "hand\n",
      "Why douruvemonton, nou whaine, arbed tu equr \n",
      "----\n",
      "iter 930000, loss: 49.413220\n",
      "*************** , with women's voices,\n",
      "St &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "Than facty has forstose our Hing lorfs herigs of these who hand heir wis hose witter mimstore bingt \n",
      "----\n",
      "iter 940000, loss: 49.508176\n",
      "*************** r measure must be answere &&&&&&&&&&&&&&&\n",
      "----\n",
      " 'd of they greend for of find lough howied tre hon that appries to the beather,\n",
      "Mundect:\n",
      "Ady you wou \n",
      "----\n",
      "iter 950000, loss: 49.240632\n",
      "*************** ting maids whose minds ar &&&&&&&&&&&&&&&\n",
      "----\n",
      " , In falawe to lave but e\n",
      "Wheted the mowell, thel.\n",
      "\n",
      "LEOL I LAUSMILO:\n",
      "Lear fe lide well, you worth my \n",
      "----\n",
      "iter 960000, loss: 49.408423\n",
      "***************  glad of this as they I c &&&&&&&&&&&&&&&\n",
      "----\n",
      " mace.\n",
      "\n",
      "RERMAPAL:\n",
      "Thee lise thouthe of live an my puulty up thanks, their thy poiss,\n",
      "For yanculon mai \n",
      "----\n",
      "iter 970000, loss: 48.929579\n",
      "***************  excess,\n",
      "And each man hav &&&&&&&&&&&&&&&\n",
      "----\n",
      " He swore well whest fing and, a vouof swou garages, viek in my light hout sterjuny: or a kon\n",
      "Kakent  \n",
      "----\n",
      "iter 980000, loss: 49.684375\n",
      "*************** love; in which cage\n",
      "of ru &&&&&&&&&&&&&&&\n",
      "----\n",
      " then I to\n",
      "gears of notthe lirve: the lais? Cank a root thee there monoull,\n",
      "What thele, ander have a  \n",
      "----\n",
      "iter 990000, loss: 49.640598\n",
      "*************** er to me more!\n",
      "\n",
      "LUCIUS:\n",
      "G &&&&&&&&&&&&&&&\n",
      "----\n",
      " the of alike me grect: Iur coopat;\n",
      "it and fang.\n",
      "IUll our but suar omert t e companden my; byought of \n",
      "----\n",
      "iter 1000000, loss: 49.803150\n",
      "*************** llus Cimber?\n",
      "\n",
      "CASSIUS:\n",
      "No &&&&&&&&&&&&&&&\n",
      "----\n",
      " dlainjech well deame.\n",
      "\n",
      "SANDICES:\n",
      "\n",
      "BATISA:\n",
      "I lewowher,\n",
      "Catious and a with whris the Sell:.\n",
      "\n",
      "PLIVCA:\n",
      "W \n",
      "----\n",
      "iter 1010000, loss: 49.182954\n",
      "*************** y,\n",
      "And heartily, for our  &&&&&&&&&&&&&&&\n",
      "----\n",
      " our su!\n",
      "For deate, not fler,\n",
      "Thusted sivem thas they crot, such work or spody, inder'r, for ou dains \n",
      "----\n",
      "iter 1020000, loss: 50.021986\n",
      "*************** was so called.\n",
      "\n",
      "NERISSA:\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " wot\n",
      "If a thoue ago, you now wis nat plie-saperageit, nons,\n",
      "I swess; as not oustorsh's duce, concoos  \n",
      "----\n",
      "iter 1030000, loss: 49.938069\n",
      "*************** gs and batty wings doth c &&&&&&&&&&&&&&&\n",
      "----\n",
      " old\n",
      "Mecan\n",
      "And and you rake michbought's foul\n",
      "supenor hid where a a stay?\n",
      "\n",
      "QEEN:\n",
      "Thes leving?\n",
      "\n",
      "SATRED \n",
      "----\n",
      "iter 1040000, loss: 49.809370\n",
      "***************  give her\n",
      "patent to offen &&&&&&&&&&&&&&&\n",
      "----\n",
      " coning ous noCIANA:\n",
      "I pitty be hang contle. my,\n",
      "Fives,\n",
      "And be loved me lolds to with too head he hom \n",
      "----\n",
      "iter 1050000, loss: 49.652613\n",
      "*************** auty will sooner\n",
      "transfor &&&&&&&&&&&&&&&\n",
      "----\n",
      " nks, aw;\n",
      "You?\n",
      "\n",
      "TRMICAUL:\n",
      "And by of,\n",
      "This sleat By your with filers you, prowit!\n",
      "\n",
      "FULICDOLLES:\n",
      "When\n",
      "I \n",
      "----\n",
      "iter 1060000, loss: 49.340459\n",
      "*************** hat claims me, one that h &&&&&&&&&&&&&&&\n",
      "----\n",
      " oud not, was then but vierons.\n",
      "\n",
      "SEWORADAET:\n",
      "SHbede. Oing re, and mecceser would omn thrabye be carce \n",
      "----\n",
      "iter 1070000, loss: 49.964470\n",
      "***************  a' would have made a\n",
      "goo &&&&&&&&&&&&&&&\n",
      "----\n",
      " nobless.\n",
      "\n",
      "graff it the eattled to corty my\n",
      "cring sutyen Hastregoot, whers spelle fings hatbed to sha \n",
      "----\n",
      "iter 1080000, loss: 49.540829\n",
      "*************** not so hardly\n",
      "As prouder  &&&&&&&&&&&&&&&\n",
      "----\n",
      " o rexss for in mew wouth the ounmann. Mormarmntte?\n",
      "Whin, be ape is to--\n",
      "Me crin:\n",
      "Them of suhis or aw \n",
      "----\n",
      "iter 1090000, loss: 49.589836\n",
      "***************  i' faith.\n",
      "\n",
      "SIR TOBY BELC &&&&&&&&&&&&&&&\n",
      "----\n",
      " ISARH:\n",
      "Heme then:\n",
      "OMILAR:\n",
      "I you shes pors,\n",
      "Any\n",
      "That sor up me.\n",
      "\n",
      "Gad I mid now puscous as\n",
      "chall the h \n",
      "----\n",
      "iter 1100000, loss: 49.388123\n",
      "*************** \n",
      "Both Citizens:\n",
      "The gods  &&&&&&&&&&&&&&&\n",
      "----\n",
      " AMICHEL:\n",
      "I thondess? all in.\n",
      "\n",
      "RIR:\n",
      "Not! food Unindes to dord, made hild I fore arred.\n",
      "\n",
      "yeavour shem! \n",
      "----\n",
      "iter 1110000, loss: 48.802304\n",
      "*************** village-cock\n",
      "Hath twice d &&&&&&&&&&&&&&&\n",
      "----\n",
      " erishis of our so Morvincy cos. Ifich\n",
      "Trome,\n",
      "And cortrigestized I strecont:\n",
      "Lake dother a it court!  \n",
      "----\n",
      "iter 1120000, loss: 49.616403\n",
      "***************  thinly scatter'd, to mak &&&&&&&&&&&&&&&\n",
      "----\n",
      " thy fands hertan on fueppaked to dis; praly perthous; I so toou aldomester:\n",
      "Inearvighars\n",
      "As be of ru \n",
      "----\n",
      "iter 1130000, loss: 49.150454\n",
      "*************** : no hearing, no feeling, &&&&&&&&&&&&&&&\n",
      "----\n",
      "  sorcht pear brcome;\n",
      "The whet thouly, que of and am-forge,\n",
      "I distiicelfed how so bud carms inter:\n",
      "O' \n",
      "----\n",
      "iter 1140000, loss: 49.138374\n",
      "*************** rs, and I have many serva &&&&&&&&&&&&&&&\n",
      "----\n",
      " d,\n",
      "I wree? That you goows thus.\n",
      "\n",
      "PELEO:\n",
      "Mather I she her swer hanserploure takes,\n",
      "And mugem make cor \n",
      "----\n",
      "iter 1150000, loss: 48.735190\n",
      "*************** e; to fear judgment; to f &&&&&&&&&&&&&&&\n",
      "----\n",
      " ever.\n",
      "\n",
      "BUTEBESK:\n",
      "I have to you, baus a that tose; for the'ld yenorucy. Do of\n",
      "I'll offake? sumeet you \n",
      "----\n",
      "iter 1160000, loss: 49.734501\n",
      "*************** \n",
      "Forbear, Seleucus.\n",
      "\n",
      "CLEO &&&&&&&&&&&&&&&\n",
      "----\n",
      " Pestian.\n",
      "\n",
      "MARDIA:\n",
      "Which!\n",
      "\n",
      "ELCHANTAREBY BENDY:\n",
      "O not not storgialrand. I PRIVIA:\n",
      "Courang:\n",
      "Beckforeat  \n",
      "----\n",
      "iter 1170000, loss: 48.546679\n",
      "*************** yal father;\n",
      "And the devic &&&&&&&&&&&&&&&\n",
      "----\n",
      "  tell huve detoserm me and be deagaves, must my lay's anter!\n",
      "\n",
      "GBET:\n",
      "Modanitt:\n",
      "Yer stans\n",
      "Why heat the \n",
      "----\n",
      "iter 1180000, loss: 49.903500\n",
      "*************** our of my noble Lord of Y &&&&&&&&&&&&&&&\n",
      "----\n",
      " ur me,\n",
      "What off staice, my bent nor im to for when a no'eld.\n",
      "\n",
      "BARD:\n",
      "Or stable of them knity thy well \n",
      "----\n",
      "iter 1190000, loss: 48.867749\n",
      "*************** r I'll speak truth.\n",
      "\n",
      "Firs &&&&&&&&&&&&&&&\n",
      "----\n",
      "  chouth and lace of, vord be the word's not to in arnetay odd to dit for of it, tthe cal sirze our p \n",
      "----\n",
      "iter 1200000, loss: 49.703776\n",
      "*************** of opinion! a man may wea &&&&&&&&&&&&&&&\n",
      "----\n",
      " r; in her?\n",
      "Sy belid the stralen awans the souse ourcepight wirnemant: be prechen'd this.\n",
      "\n",
      "CKEMICD:\n",
      "W \n",
      "----\n",
      "iter 1210000, loss: 49.617684\n",
      "***************  whetstone of your sword: &&&&&&&&&&&&&&&\n",
      "----\n",
      " his woalt is inennomes that oure the beging!-be deftle, off arse,\n",
      "I seack Heress swall before ance d \n",
      "----\n",
      "iter 1220000, loss: 49.898213\n",
      "*************** in, I\n",
      "hate the Moor: my c &&&&&&&&&&&&&&&\n",
      "----\n",
      " ld. When you say, ver heath seak, byee,\n",
      "If mading sersende laothame, the as Gage, and undex, feartus \n",
      "----\n",
      "iter 1230000, loss: 49.791674\n",
      "*************** o!\n",
      "\n",
      "BERNARDO:\n",
      "Say,\n",
      "What,  &&&&&&&&&&&&&&&\n",
      "----\n",
      " utred'tes, bortind by,\n",
      "Nit not acialt of my lore of than\n",
      "Mall ant:\n",
      "As,\n",
      "On worlt thus offead ongacy,\n",
      " \n",
      "----\n",
      "iter 1240000, loss: 49.526866\n",
      "*************** et your 'quies,'\n",
      "your 'qu &&&&&&&&&&&&&&&\n",
      "----\n",
      " vognd that dock thee, ar mustry fatis lovady end thee.\n",
      "\n",
      "CHELY:\n",
      "Sile I hime? that nore\n",
      "Bel!\n",
      "\n",
      "PHERTIA: \n",
      "----\n",
      "iter 1250000, loss: 49.715729\n",
      "*************** ate on blood,--\n",
      "To sweari &&&&&&&&&&&&&&&\n",
      "----\n",
      " n,-\n",
      "Whisbaker,\n",
      "Mante be grastion the bath ip and keave, that seftacion\n",
      "'s rects.\n",
      "\n",
      "MAGELIUS:\n",
      "Exesping \n",
      "----\n",
      "iter 1260000, loss: 49.589673\n",
      "*************** te noble master!\n",
      "\n",
      "TIMON:\n",
      " &&&&&&&&&&&&&&&\n",
      "----\n",
      " hee all inder!\n",
      "That hery reetung novess and nous sirrands that vies tirlh wis net moves parter-send' \n",
      "----\n",
      "iter 1270000, loss: 49.487820\n",
      "*************** ey the Great; and Suffolk &&&&&&&&&&&&&&&\n",
      "----\n",
      "  upon and vinf conensters, hand thope for plise'd rase As, be would shall in your ar, I sstose the d \n",
      "----\n",
      "iter 1280000, loss: 49.090328\n",
      "*************** ts and prisons, and to th &&&&&&&&&&&&&&&\n",
      "----\n",
      "  ho prings gair of calound! I fetty anstedry whou will in in his coup det the It T curnered sir to m \n",
      "----\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter 1290000, loss: 48.361000\n",
      "*************** d with thee?\n",
      "\n",
      "Pursuivant: &&&&&&&&&&&&&&&\n",
      "----\n",
      " ing nem, this brerpa noble not youre to my fool so and thath, abore broonem.\n",
      "\n",
      "BRUVBELLUS CAR:\n",
      "Have b \n",
      "----\n",
      "iter 1300000, loss: 49.029042\n",
      "*************** this, being smelt, with t &&&&&&&&&&&&&&&\n",
      "----\n",
      " he:\n",
      "Gereces is of thou a me in in frows ponem bo vaite,\n",
      "I bel!\n",
      "Yet'st I fat and\n",
      "That Cothersord'd in \n",
      "----\n",
      "iter 1310000, loss: 48.605222\n",
      "*************** ERMIONE:\n",
      "What is this? sp &&&&&&&&&&&&&&&\n",
      "----\n",
      " MILEO:\n",
      "I sake sould, she bidy teath.\n",
      "\n",
      "ORTAFF:\n",
      "In eates-\n",
      "\n",
      "TICKENDIAS OF'ZOLINK:\n",
      "Whys, thou don nch th \n",
      "----\n",
      "iter 1320000, loss: 48.623138\n",
      "*************** maid at home;\n",
      "And therefo &&&&&&&&&&&&&&&\n",
      "----\n",
      " es I faresilver wanbowartit thee giple spart,\n",
      "Ay; onar I whenour have feibany a with deen.\n",
      "\n",
      "ETHUMI:\n",
      " \n",
      "----\n",
      "iter 1330000, loss: 48.844429\n",
      "*************** they say.\n",
      "\n",
      "MARGARET:\n",
      "By m &&&&&&&&&&&&&&&\n",
      "----\n",
      " so him onembeary in whangs havill, be ey, and ch.\n",
      "I honour, her. mumbad? Gove see\n",
      "And you have band  \n",
      "----\n",
      "iter 1340000, loss: 49.257849\n",
      "***************  off: our graver business &&&&&&&&&&&&&&&\n",
      "----\n",
      " Prob:\n",
      "Maltuso an--yis her scomeforsongep theng theup.\n",
      "\n",
      "CLOUS YIZ:\n",
      "That, in mest they hom not wheaven \n",
      "----\n",
      "iter 1350000, loss: 48.420602\n",
      "*************** o as lightly,\n",
      "Himself wou &&&&&&&&&&&&&&&\n",
      "----\n",
      " .\n",
      "UCHINGO:\n",
      "For inswathans-mesher?\n",
      "Sit\n",
      "O you meer waghur the so see fing cas in the lispare in a forc \n",
      "----\n",
      "iter 1360000, loss: 49.000519\n",
      "*************** enge\n",
      "These wrongs, unspea &&&&&&&&&&&&&&&\n",
      "----\n",
      "  tuesir. Bet frinnoute is dighire and hen beact'g'd ERSABOBAND:\n",
      "I frly: of or wilsest there's throme \n",
      "----\n",
      "iter 1370000, loss: 48.871234\n",
      "*************** ge, may lawfully\n",
      "make tit &&&&&&&&&&&&&&&\n",
      "----\n",
      " ? withsiro, Cloce.\n",
      "\n",
      "Marcestersied. so deack I bady:\n",
      "More:\n",
      "Alen besie, the wonce a badkean\n",
      "Se ford;\n",
      "A \n",
      "----\n",
      "iter 1380000, loss: 48.962530\n",
      "*************** r comes Paris, yonder com &&&&&&&&&&&&&&&\n",
      "----\n",
      " amisalang: will dofounter thet youufo bext thou hear this hould. What what knoh we sace.\n",
      "\n",
      "BRETAS:\n",
      "Wh \n",
      "----\n",
      "iter 1390000, loss: 49.190493\n",
      "*************** , and I'll do.\n",
      "\n",
      "Second Wi &&&&&&&&&&&&&&&\n",
      "----\n",
      "  thend hor worn, I him this forming of a had's honow\n",
      "Any hose'er my the by forthe as thy profeth\n",
      "He  \n",
      "----\n",
      "iter 1400000, loss: 50.183932\n",
      "*************** w, understanding that the &&&&&&&&&&&&&&&\n",
      "----\n",
      "  heres\n",
      "at shausarard we hance yourenses the sterchusw, with my rit qeafure of nowly,\n",
      "As to sir, one; \n",
      "----\n",
      "iter 1410000, loss: 49.492534\n",
      "*************** ge, my masters.\n",
      "\n",
      "PRINCE H &&&&&&&&&&&&&&&\n",
      "----\n",
      " his hon as spevrans hild-mas, yould ver. I, maroved an yaly forlant is shall his me how with hour di \n",
      "----\n",
      "iter 1420000, loss: 48.927426\n",
      "***************  you say, 'Marry her,' I  &&&&&&&&&&&&&&&\n",
      "----\n",
      " face a wellot'rcouses her o'er heve,\n",
      "U throved is bage, queestrogit comest, her an; ap will amy feou \n",
      "----\n",
      "iter 1430000, loss: 49.082407\n",
      "*************** t de nails.\n",
      "\n",
      "ALICE:\n",
      "C'est &&&&&&&&&&&&&&&\n",
      "----\n",
      " airs sakb oaly: took butdury there iss new\n",
      "I the kin's Naw\n",
      "mand in that mior Fangu beirs's wit say a \n",
      "----\n",
      "iter 1440000, loss: 49.173241\n",
      "***************  then thou mightst kill ' &&&&&&&&&&&&&&&\n",
      "----\n",
      " slus you so dediasio, imfict; my lost, mef colame, and wirn's any you, rthe, 'dow I knann.\n",
      "Hight Slu \n",
      "----\n",
      "iter 1450000, loss: 49.377571\n",
      "*************** ak.\n",
      "\n",
      "WARWICK:\n",
      "The cardina &&&&&&&&&&&&&&&\n",
      "----\n",
      " n\n",
      "on twenigniey, from Cleads, fee his terward.\n",
      "\n",
      "PIUS ALIO:\n",
      "WhactEShe Inovouserinss of yet\n",
      "iten lord, \n",
      "----\n",
      "iter 1460000, loss: 48.905774\n",
      "***************  and o'erswell\n",
      "With cours &&&&&&&&&&&&&&&\n",
      "----\n",
      " sweed the have,\n",
      "So be why my heart sbe\n",
      "Thel!\n",
      "\n",
      "I hosted or moner.\n",
      "\n",
      "MARMIANY:\n",
      "That or khat that saund  \n",
      "----\n",
      "iter 1470000, loss: 48.400553\n",
      "*************** TER:\n",
      "Stay, you that bear  &&&&&&&&&&&&&&&\n",
      "----\n",
      " hat,\n",
      "And the deten! Cagerey,\n",
      "I I't whatilk stail; wesh thus hetiss all nevark me un\n",
      "precoy\n",
      "To she we \n",
      "----\n",
      "iter 1480000, loss: 48.705249\n",
      "*************** ous article,\n",
      "Containing t &&&&&&&&&&&&&&&\n",
      "----\n",
      "  alume;\n",
      "Anis stain,\n",
      "To fringed wary Iarr; rice.\n",
      "\n",
      "Well great igurs.\n",
      "\n",
      "GLOUMIRY:\n",
      "Capes of is fank to is \n",
      "----\n",
      "iter 1490000, loss: 48.544493\n",
      "*************** nd ourselves;\n",
      "In them and &&&&&&&&&&&&&&&\n",
      "----\n",
      " ot is now whok i! conded erpbeasiop\n",
      "AUS neattern heaw crown corkincies stort blo; so? Frown a sivid! \n",
      "----\n",
      "iter 1500000, loss: 48.475490\n",
      "*************** ieve I know the cause of  &&&&&&&&&&&&&&&\n",
      "----\n",
      " t, this pafts,\n",
      "Of an with, my el, but loigitiod like\n",
      "Cle:\n",
      "And but me deave thers, now,\n",
      "Where, what m \n",
      "----\n",
      "iter 1510000, loss: 49.049299\n",
      "*************** wish\n",
      "Myself were mudded i &&&&&&&&&&&&&&&\n",
      "----\n",
      " om the deeard up yeathen but there tir lived\n",
      "With thy majem\n",
      "A'e glotern;\n",
      "Shought erosution, I home o \n",
      "----\n",
      "iter 1520000, loss: 48.588939\n",
      "*************** ver see his pardon; for m &&&&&&&&&&&&&&&\n",
      "----\n",
      " er wioh then good gicling they; lighton and not they alose my lotge now not ling\n",
      "The pure\n",
      "'Tost I ya \n",
      "----\n",
      "iter 1530000, loss: 48.785482\n",
      "*************** prey on nothing that doth &&&&&&&&&&&&&&&\n",
      "----\n",
      " ur a shinkbe tall ind woow goor he woul had him.\n",
      "\n",
      "ROYFLED:\n",
      "Whinch-were of not of hy, const\n",
      "Rome:\n",
      "You \n",
      "----\n",
      "iter 1540000, loss: 49.147040\n",
      "***************  warrant you, madam, we w &&&&&&&&&&&&&&&\n",
      "----\n",
      " that vay, in trus.\n",
      "\n",
      "Lempopk sur,\n",
      "Loocel loty, redairs with shinge;\n",
      "Peries\n",
      "Let of pery reporamon and  \n",
      "----\n",
      "iter 1550000, loss: 49.191889\n",
      "*************** .\n",
      "\n",
      "DECIUS BRUTUS:\n",
      "What, s &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "\n",
      "SIG BELINCE:\n",
      "You sear be are is, this parazen tarrer at and tisuragath some this and then, pruugat \n",
      "----\n",
      "iter 1560000, loss: 48.538296\n",
      "*************** \n",
      "Believe it.\n",
      "\n",
      "SURREY:\n",
      "Wil &&&&&&&&&&&&&&&\n",
      "----\n",
      " vene force,\n",
      "IXERUS:\n",
      "To unochad uch wather and fith'd sceand feeblemose'ld papalpun\n",
      "End goverd!\n",
      "\n",
      "Clow \n",
      "----\n",
      "iter 1570000, loss: 49.502420\n",
      "*************** t here an angel in a gold &&&&&&&&&&&&&&&\n",
      "----\n",
      " y?\n",
      "\n",
      "CIRNIS:\n",
      "Then wheirferty ant cid Orone; re what witt he and as hath\n",
      "The er desce then.\n",
      "This arsmo \n",
      "----\n",
      "iter 1580000, loss: 49.493480\n",
      "***************  world's desires,--\n",
      "Our l &&&&&&&&&&&&&&&\n",
      "----\n",
      " jastaver! es qugity, ase of from?\n",
      "Maughter befaning know fall,\n",
      "The, thoufly, and nothat as det gack, \n",
      "----\n",
      "iter 1590000, loss: 48.800587\n",
      "*************** sio kill'd!\n",
      "\n",
      "EMILIA:\n",
      "No,  &&&&&&&&&&&&&&&\n",
      "----\n",
      " atle deartrow eyed, Caunementer, I in.\n",
      "When to truutly shas ick reass it Here our deabitather fith m \n",
      "----\n",
      "iter 1600000, loss: 48.791842\n",
      "***************  passion, lets go by\n",
      "The  &&&&&&&&&&&&&&&\n",
      "----\n",
      " hath uncom?\n",
      "\n",
      "LOLIO:\n",
      "Tet, have,\n",
      "That pirnother's,\n",
      "what belf If a tromes moopeghthe can host bidewh mi \n",
      "----\n",
      "iter 1610000, loss: 48.327720\n",
      "*************** t was sauced with thy upb &&&&&&&&&&&&&&&\n",
      "----\n",
      " os st it a mand the larent in verther; I wich thy day; it Of deat your will,\n",
      "And at gright het uf th \n",
      "----\n",
      "iter 1620000, loss: 49.353401\n",
      "*************** g?\n",
      "What peer hath been su &&&&&&&&&&&&&&&\n",
      "----\n",
      " ht bor?\n",
      "\n",
      "FALSTALY:\n",
      "O Cames goo'ans!\n",
      "O no worlt illan: evendelangllies off coups betces then a likns  \n",
      "----\n",
      "iter 1630000, loss: 49.001687\n",
      "*************** wear not\n",
      "My dagger in my  &&&&&&&&&&&&&&&\n",
      "----\n",
      " akes and now's tind.\n",
      "\n",
      "MISTAGLAy: this here dis,\n",
      "Wher if proodar;\n",
      "Of to to heard Spour yet it he: to  \n",
      "----\n",
      "iter 1640000, loss: 48.879343\n",
      "*************** .\n",
      "\n",
      "SIR TOBY BELCH:\n",
      "Look,  &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "\n",
      "FURDES:\n",
      "O mut for Sursan braver,\n",
      "But I d Juk blo, ter Sall-dine\n",
      "I\n",
      "prield sion.\n",
      "Vere'd ll redie! So \n",
      "----\n",
      "iter 1650000, loss: 48.447797\n",
      "*************** , I\n",
      "Will answer in mine h &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "Come:\n",
      "Rised?---veristhes, aid en the it the crows. Feath for hatou bisaneman,\n",
      "So us, ime; speest du \n",
      "----\n",
      "iter 1660000, loss: 48.687822\n",
      "*************** HARD II:\n",
      "Why uncle, thou  &&&&&&&&&&&&&&&\n",
      "----\n",
      " astan my lond-y\n",
      "Blait is him lidours:\n",
      "Tooly lovemob, as and I them; when then plaer?\n",
      "\n",
      "LORI HAMILLEUD \n",
      "----\n",
      "iter 1670000, loss: 48.915229\n",
      "*************** er play the orator.\n",
      "\n",
      "MONT &&&&&&&&&&&&&&&\n",
      "----\n",
      " nd;\n",
      "And set gell Or seign'd trectrepalt: bray'lt\n",
      "I madnce but in take; and you mece no-\n",
      "Thaid tand?  \n",
      "----\n",
      "iter 1680000, loss: 48.575945\n",
      "*************** aughter came to look upon &&&&&&&&&&&&&&&\n",
      "----\n",
      " nck, say, my your so, and thee aughind greick?\n",
      "Shall me,\n",
      "\n",
      "nome silcies a tloughter's, near:\n",
      "I stairy \n",
      "----\n",
      "iter 1690000, loss: 48.409992\n",
      "***************  for the purpose hurried  &&&&&&&&&&&&&&&\n",
      "----\n",
      " toist a genels my you, a pread journ ruughts:\n",
      "I hill me.\n",
      "\n",
      "Fid thy Yeady memly, I wlo.\n",
      "\n",
      "IICELRO:\n",
      "Thic \n",
      "----\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iter 1700000, loss: 48.131510\n",
      "*************** AR:\n",
      "'Tis strange that the &&&&&&&&&&&&&&&\n",
      "----\n",
      " , thed hasts or ough doth a if Eccurgo welle's. Yet ugave his man younge\n",
      "With hear, thel in I son.\n",
      "\n",
      " \n",
      "----\n",
      "iter 1710000, loss: 49.060366\n",
      "*************** tions.\n",
      "\n",
      "ROSALIND:\n",
      "O, they &&&&&&&&&&&&&&&\n",
      "----\n",
      " ,\n",
      "Her!\n",
      "No the guuld, Orcefore, an the praid\n",
      "\n",
      "Yet hip in thy whiment; shrean red cought\n",
      "Fit.\n",
      "\n",
      "I will  \n",
      "----\n",
      "iter 1720000, loss: 48.379484\n",
      "*************** once again; reserve\n",
      "That  &&&&&&&&&&&&&&&\n",
      "----\n",
      " f in that whelf but males that has for to but madam\n",
      "The paluchtuly atsllan?\n",
      "\n",
      "Secour po but sinds; he \n",
      "----\n",
      "iter 1730000, loss: 49.453309\n",
      "***************  to help me now.\n",
      "No hope  &&&&&&&&&&&&&&&\n",
      "----\n",
      " trould knool Slear, there Jaiz:\n",
      "I'll t\n",
      "The den fiving your frided good knose offeewarres,\n",
      "But dle my \n",
      "----\n",
      "iter 1740000, loss: 47.985924\n",
      "*************** e married: but thou art t &&&&&&&&&&&&&&&\n",
      "----\n",
      "  a her care and not brebitlitycy\n",
      "Out thy nearreather.\n",
      "\n",
      "Creepinge molairs.\n",
      "My chantas.\n",
      "\n",
      "Tim he's ase. \n",
      "----\n",
      "iter 1750000, loss: 49.230498\n",
      "*************** s of Greece, go to my ten &&&&&&&&&&&&&&&\n",
      "----\n",
      "  aleak\n",
      "And sor he preppen Sirm; wrilopper: Canendly t,\n",
      "If thou it the ma frind fall bess ind unford  \n",
      "----\n",
      "iter 1760000, loss: 49.301512\n",
      "*************** . 'Thisne,\n",
      "Thisne;' 'Ah,  &&&&&&&&&&&&&&&\n",
      "----\n",
      " \n",
      "\n",
      "Foll dish'd ill fort\n",
      "Thrimin Menick!\n",
      "Dy.\n",
      "\n",
      "COROT: natithen stroight; a be spoun lough when than!\n",
      "\n",
      "C \n",
      "----\n",
      "iter 1770000, loss: 49.347304\n",
      "*************** e free elements. And then &&&&&&&&&&&&&&&\n",
      "----\n",
      "  pucrowit.\n",
      "\n",
      "Clomises a celed that will nave to gooved will, Sigation peatise: wewnesse worth amorsw  \n",
      "----\n",
      "iter 1780000, loss: 48.924169\n",
      "*************** MLET:\n",
      "Murder!\n",
      "\n",
      "Ghost:\n",
      "Mur &&&&&&&&&&&&&&&\n",
      "----\n",
      " OS:\n",
      "\n",
      "FALINCENRENS:\n",
      "Thou have dwise a\n",
      "Pust mart that were my dia either, as he mayou barving Siferick \n",
      "----\n",
      "iter 1790000, loss: 48.871540\n",
      "*************** \n",
      "Pinch him, fairies, mutu &&&&&&&&&&&&&&&\n",
      "----\n",
      " Who good to com.\n",
      "\n",
      "MESTRADY:\n",
      "Fracles knce that licellughtertud, regroge, my ungedee disty to agar\n",
      "I t \n",
      "----\n",
      "iter 1800000, loss: 49.377017\n",
      "*************** e:\n",
      "Sir, here comes the no &&&&&&&&&&&&&&&\n",
      "----\n",
      " d tild, of fiving\n",
      "As be quobld acceate?\n",
      "Shim ald w\n",
      "Up prenevires the pas\n",
      "'Word thee and ever sound i \n",
      "----\n",
      "iter 1810000, loss: 48.964035\n",
      "*************** be\n",
      "better acquainted.\n",
      "\n",
      "IA &&&&&&&&&&&&&&&\n",
      "----\n",
      " lestat:\n",
      "And then end sid the ast valtzous fore awaors costy: my iebouth'd the breack sten: When such \n",
      "----\n",
      "iter 1820000, loss: 48.893967\n",
      "*************** e to merit such a bounty. &&&&&&&&&&&&&&&\n",
      "----\n",
      " nd to triefe parst thus ip shalblew thidlting as sweetn Lard! BEOTRUS:\n",
      "Lerand beath.\n",
      "\n",
      "\n",
      "ICHERLAUS:\n",
      "Ad \n",
      "----\n",
      "iter 1830000, loss: 48.783877\n",
      "*************** is bloody brow\n",
      "With his m &&&&&&&&&&&&&&&\n",
      "----\n",
      " rerp he\n",
      "makent\n",
      "My rend thy hands all hing for way, my courn:\n",
      "fore a body it courty rompe we\n",
      "mean, wh \n",
      "----\n",
      "iter 1840000, loss: 47.761218\n",
      "*************** erate eyes:\n",
      "High-reaching &&&&&&&&&&&&&&&\n",
      "----\n",
      " t in some's ace yould to seath gextince, I sing;\n",
      "Whose?\n",
      "\n",
      "BESTRICK:\n",
      "The fithill here an lod to bulaur \n",
      "----\n",
      "iter 1850000, loss: 48.876411\n",
      "*************** urse:\n",
      "It did, it did; ala &&&&&&&&&&&&&&&\n",
      "----\n",
      " th he your lanabeser'st kned thou happow for a pluois, know yet:\n",
      "Be alse; hon Tasy. Rertersle on nd  \n",
      "----\n",
      "iter 1860000, loss: 48.089011\n",
      "***************  causes of their death ap &&&&&&&&&&&&&&&\n",
      "----\n",
      " your deech dis tou love hes I GORLAUDE Panter, on fapen king wor utbo,\n",
      "To fought, ferich wive in mow \n",
      "----\n",
      "iter 1870000, loss: 48.247652\n",
      "***************  so fast, protesting oath &&&&&&&&&&&&&&&\n",
      "----\n",
      " my counimus nd am to perthe he, in butt line ture, head, this begranner the\n",
      "por f cound by noxtelued \n",
      "----\n",
      "iter 1880000, loss: 48.307950\n",
      "*************** thy wit?\n",
      "\n",
      "BENEDICK:\n",
      "It is &&&&&&&&&&&&&&&\n",
      "----\n",
      " oon of a so, then you, though youl surpow.\n",
      "\n",
      "ARIZO:\n",
      "Will mast all say, I hat a dure be precered do mu \n",
      "----\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)\n",
    "BSD License\n",
    "\"\"\"\n",
    "\n",
    "import numpy as np\n",
    "import random\n",
    "# data I/O\n",
    "progress_step = 10000\n",
    "data = open('shakespeare_input.txt', 'r').read() # should be simple plain text file\n",
    "chars = list(set(data))\n",
    "data_size, vocab_size = len(data), len(chars)\n",
    "print('data has %d characters, %d unique.' % (data_size, vocab_size))\n",
    "      \n",
    "char_to_ix = { ch:i for i,ch in enumerate(chars) }\n",
    "ix_to_char = { i:ch for i,ch in enumerate(chars) }\n",
    "# hyperparameters\n",
    "hidden_size = 100 # size of hidden layer of neurons\n",
    "seq_length = 25 # number of steps to unroll the RNN for\n",
    "learning_rate = 1e-1\n",
    "# model parameters\n",
    "Wxh = np.random.randn(hidden_size, vocab_size)*0.01 # input to hidden\n",
    "Whh = np.random.randn(hidden_size, hidden_size)*0.01 # hidden to hidden\n",
    "Why = np.random.randn(vocab_size, hidden_size)*0.01 # hidden to output\n",
    "bh = np.zeros((hidden_size, 1)) # hidden bias\n",
    "by = np.zeros((vocab_size, 1)) # output bias\n",
    "def lossFun(inputs, targets, hprev):\n",
    "    \"\"\"\n",
    "    inputs,targets are both list of integers.\n",
    "    hprev is Hx1 array of initial hidden state\n",
    "    returns the loss, gradients on model parameters, and last hidden state\n",
    "    \"\"\"\n",
    "    xs, hs, ys, ps = {}, {}, {}, {}\n",
    "    hs[-1] = np.copy(hprev)\n",
    "    loss = 0\n",
    "    # forward pass\n",
    "    for t in range(len(inputs)):\n",
    "        xs[t] = np.zeros((vocab_size,1)) # encode in 1-of-k representation\n",
    "        xs[t][inputs[t]] = 1\n",
    "        hs[t] = np.tanh(np.dot(Wxh, xs[t]) + np.dot(Whh, hs[t-1]) + bh) # hidden state\n",
    "        ys[t] = np.dot(Why, hs[t]) + by # unnormalized log probabilities for next chars\n",
    "        ps[t] = np.exp(ys[t]) / np.sum(np.exp(ys[t])) # probabilities for next chars\n",
    "        loss += -np.log(ps[t][targets[t],0]) # softmax (cross-entropy loss)\n",
    "    # backward pass: compute gradients going backwards\n",
    "    dWxh, dWhh, dWhy = np.zeros_like(Wxh), np.zeros_like(Whh), np.zeros_like(Why)\n",
    "    dbh, dby = np.zeros_like(bh), np.zeros_like(by)\n",
    "    dhnext = np.zeros_like(hs[0])\n",
    "    for t in reversed(range(len(inputs))):\n",
    "        dy = np.copy(ps[t])\n",
    "        dy[targets[t]] -= 1 # backprop into y. see http://cs231n.github.io/neural-networks-case-study/#grad if confused here\n",
    "        dWhy += np.dot(dy, hs[t].T)\n",
    "        dby += dy\n",
    "        dh = np.dot(Why.T, dy) + dhnext # backprop into h\n",
    "        dhraw = (1 - hs[t] * hs[t]) * dh # backprop through tanh nonlinearity\n",
    "        dbh += dhraw\n",
    "        dWxh += np.dot(dhraw, xs[t].T)\n",
    "        dWhh += np.dot(dhraw, hs[t-1].T)\n",
    "        dhnext = np.dot(Whh.T, dhraw)\n",
    "    for dparam in [dWxh, dWhh, dWhy, dbh, dby]:\n",
    "        np.clip(dparam, -5, 5, out=dparam) # clip to mitigate exploding gradients\n",
    "    return loss, dWxh, dWhh, dWhy, dbh, dby, hs[len(inputs)-1]\n",
    "\n",
    "\n",
    "def sample(h, seed_ix, n):\n",
    "    \"\"\" \n",
    "    sample a sequence of integers from the model \n",
    "    h is memory state, seed_ix is seed letter for first time step\n",
    "    \"\"\"\n",
    "    x = np.zeros((vocab_size, 1))\n",
    "    x[seed_ix] = 1\n",
    "    ixes = []\n",
    "    for t in range(n):\n",
    "        h = np.tanh(np.dot(Wxh, x) + np.dot(Whh, h) + bh)\n",
    "        y = np.dot(Why, h) + by\n",
    "        p = np.exp(y) / np.sum(np.exp(y))\n",
    "        ix = np.random.choice(range(vocab_size), p=p.ravel())\n",
    "        x = np.zeros((vocab_size, 1))\n",
    "        x[ix] = 1\n",
    "        ixes.append(ix)\n",
    "    return ixes\n",
    "\n",
    "\n",
    "n, p = 0, 0\n",
    "mWxh, mWhh, mWhy = np.zeros_like(Wxh), np.zeros_like(Whh), np.zeros_like(Why)\n",
    "mbh, mby = np.zeros_like(bh), np.zeros_like(by) # memory variables for Adagrad\n",
    "smooth_loss = -np.log(1.0/vocab_size)*seq_length # loss at iteration 0\n",
    "while True:\n",
    "    # prepare inputs (we're sweeping from left to right in steps seq_length long)\n",
    "    if p+seq_length+1 >= len(data) or n == 0: \n",
    "        hprev = np.zeros((hidden_size,1)) # reset RNN memory\n",
    "        p = 0 # go from start of data\n",
    "    inputs = [char_to_ix[ch] for ch in data[p:p+seq_length]]\n",
    "    targets = [char_to_ix[ch] for ch in data[p+1:p+seq_length+1]]\n",
    "    # forward seq_length characters through the net and fetch gradient\n",
    "    loss, dWxh, dWhh, dWhy, dbh, dby, hprev = lossFun(inputs, targets, hprev)\n",
    "    smooth_loss = smooth_loss * 0.999 + loss * 0.001\n",
    "    if n % progress_step == 0: \n",
    "        print('iter %d, loss: %f' % (n, smooth_loss)) # print progress\n",
    "        if  smooth_loss < 55:\n",
    "            # sample from the model now and then\n",
    "            print(\"*\"*15,''.join(ix_to_char[ix] for ix in inputs),\"&\"*15)\n",
    "            sample_ix = sample(hprev, inputs[0], 100)\n",
    "            txt = ''.join(ix_to_char[ix] for ix in sample_ix)\n",
    "            print('----\\n %s \\n----' % (txt, ))\n",
    "\n",
    "\n",
    "        if smooth_loss < 0.01:\n",
    "            break\n",
    "        # perform parameter update with Adagrad\n",
    "    for param, dparam, mem in zip([Wxh, Whh, Why, bh, by], \n",
    "                                [dWxh, dWhh, dWhy, dbh, dby], \n",
    "                                [mWxh, mWhh, mWhy, mbh, mby]):\n",
    "        mem += dparam * dparam\n",
    "        param += -learning_rate * dparam / np.sqrt(mem + 1e-8) # adagrad update\n",
    "        p += seq_length # move data pointer\n",
    "        n += 1 # iteration counter "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}