ch11-regexes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regular expressions (Chap 11)\n",
    "- Regular expressions appear in many programming languages, with minor differences among the incarnations. Their purpose is to specify character patterns that subsequently are determined to match (or not match) strings. Pattern matching, in turn, serves as the basis for operations like parsing log files, testing keyboard input for validity, and isolating substrings—operations, in other words, of frequent and considerable use to anyone who has to process strings and text.\n",
    "\n",
    "- Regular expressions have a weird reputation. Using them is a powerful, concentrated technique; they burn through a large subset of text-processing problems like acid through a padlock. They’re also, in the view of many people (including people who understand them well), difficult to use, difficult to read, opaque,  unmaintainable, and ultimately counterproductive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Writing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# seeing patterns (11.2.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Regexp\n",
      "String\n",
      "true\n",
      "true\n",
      "#<MatchData \"abc\">\n",
      "nil\n"
     ]
    }
   ],
   "source": [
    "# simple matches (11.2.2)\n",
    "\n",
    "p //.class # constructor\n",
    "p %{}.class # alias\n",
    "p /abc/.match?(\"The alphabet starts with abc.\")\n",
    "p \"The alphabet starts with abc.\".match?(/abc/)\n",
    "p /abc/.match(\"The alphabet starts with abc.\")\n",
    "p /abc/.match(\"def\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "25"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# =~ returns index of character in string where match occurred\n",
    "p \"The alphabet starts with abc\" =~ /abc/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![regex patterns](px/Selection_164.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Building a pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# literal character patterns (11.3.1)\n",
    "p \"The alphabet starts with abc\" =~ /a/ # 1st match of \"a\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "true\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# wildcards (11.3.2)\n",
    "<<READ\n",
    "Sometimes you’ll want to match any character at some point in your pattern. You do\n",
    "this with the special dot wildcard character (.). A dot matches any character with the\n",
    "exception of a newline.\n",
    "READ\n",
    "    \n",
    "p /.ejected/.match?(\"%ejected\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n",
      "\" is a hex number.\"\n",
      "\"ABC3934\"\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"ABC3934\""
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# character classes (11.3.3)\n",
    "<<READ \n",
    "A character class is an explicit list of characters placed inside the regexp in square\n",
    "brackets. Example matchers:\n",
    "lowercase letters: /[a-z]/\n",
    "hex digits: /[A-Fa-f0-9]/\n",
    "negative hex digits: [^A-Fa-f0-9]\n",
    "digits: /[0-9]/\n",
    "READ\n",
    "\n",
    "string = \"ABC3934 is a hex number.\"\n",
    "p string =~ %r{[^A-Fa-f0-9]}\n",
    "p string[7..-1]\n",
    "p string[0...7]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"  /d/ = any digit\\n  /w/ = any digit, alphanumerical or underscore\\n  / / = any whitespace\\n  /D/ = any non-digit\\n  /W/ = any non-alphanumeric character or underscore\\n  /S/ = non non-whitespace\\n\""
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# escape sequences for common character classes\n",
    "<<READ\n",
    "  /\\d/ = any digit\n",
    "  /\\w/ = any digit, alphanumerical or underscore\n",
    "  /\\s/ = any whitespace\n",
    "  /\\D/ = any non-digit\n",
    "  /\\W/ = any non-alphanumeric character or underscore\n",
    "  /\\S/ = non non-whitespace\n",
    "READ"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Matching, substrings, MatchData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#<MatchData \"Peel,Emma,Mrs.\" 1:\"Peel\" 2:\"Mrs.\">\n",
      "\"Dear Mrs., Peel,\"\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Dear Mrs., Peel,\""
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# capturing submatches (11.4.1)\n",
    "\n",
    "<<READ\n",
    "When we perform the match, 1) we get a MatchData object that gives \n",
    "us access to the submatches (discussed in a moment). 2) Ruby \n",
    "automatically populates a series of variables for us, which also \n",
    "give us access to those submatches. The variables are global;\n",
    "their names are based on numbers: $1 , $2 , and so forth. \n",
    "READ\n",
    "\n",
    "s = \"Peel,Emma,Mrs.,talented amateur\"\n",
    "p /([A-Za-z]+),[A-Za-z]+,(Mrs?\\.)/.match(\n",
    "    \"Peel,Emma,Mrs.,talented amateur\")\n",
    "p \"Dear #{$2}, #{$1},\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nil\n",
      "start: My phone number is (123) 555-1234.\n",
      "entire matching part: (123) 555-1234\n",
      "three captures: \n",
      "123\n",
      "555\n",
      "1234\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# match success & failure (11.4.2)\n",
    "\n",
    "p %r{a}.match(\"b\") # non-matches return nil\n",
    "\n",
    "<<READ\n",
    "The MatchData object returned by a successful match has a \n",
    "Boolean value of true. You must first save it.\n",
    "READ\n",
    "\n",
    "string   = \"My phone number is (123) 555-1234.\"\n",
    "phone_re = %r{\\((\\d{3})\\)\\s+(\\d{3})-(\\d{4})}\n",
    "m        = phone_re.match(string)\n",
    "unless m\n",
    "    puts \"no match. sorry.\"; exit\n",
    "end\n",
    "puts \"start: \"+m.string\n",
    "puts \"entire matching part: \"+m[0]\n",
    "puts \"three captures: \"\n",
    "3.times do |ndx|\n",
    "    puts \"#{m.captures[ndx]}\"\n",
    "end\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# two ways of getting captures (11.4.3)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "David A. Black\n",
      "David\n",
      "{\"first\"=>\"David\", \"middle\"=>\"A.\", \"last\"=>\"Black\"}\n"
     ]
    }
   ],
   "source": [
    "# named captures (in this case, <first>,<middle>,<last>)\n",
    "s = \"David A. Black\"\n",
    "re = %r{(?<first>\\w+)\\s+((?<middle>\\w\\.)\\s+)(?<last>\\w+)}\n",
    "puts m  = re.match(s)\n",
    "puts m[:first]\n",
    "puts m.named_captures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "6\n",
      "14\n"
     ]
    }
   ],
   "source": [
    "# other MatchData info (11.4.4)\n",
    "puts m.pre_match\n",
    "puts m.post_match\n",
    "puts m.begin(2) # 2nd capture, beginning char#\n",
    "puts m.end(3) # 3rd capture, ending char#"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Quantifiers, anchors & modifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(?-mix:Mrs?\\.?)\n"
     ]
    }
   ],
   "source": [
    "# constraining matches with quantifiers (11.5.1)\n",
    "<<READ\n",
    "Regexp syntax gives you ways to specify not only what you want \n",
    "but also how many: exactly one of a particular character, \n",
    "5–10 repetitions of a subpattern, and so forth.\n",
    "READ"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "abc!def!ghi!\n",
      "abc!\n"
     ]
    }
   ],
   "source": [
    "# greedy & non-greedy quantifiers (11.5.2)\n",
    "<<READ\n",
    "The * (zero-or-more) and + (one-or-more) quantifiers are greedy. \n",
    "They match as many characters as possible, consistent with \n",
    "allowing the rest of the pattern to match.\n",
    "READ\n",
    "\n",
    "string = \"abc!def!ghi!\"\n",
    "puts /.+!/.match(string)[0]\n",
    "\n",
    "<<READ\n",
    "We’ve asked for one or more characters (using the wildcard dot) \n",
    "followed by an exclamation point. You might expect to get back \n",
    "the substring \"abc!\" , which fits that description.\n",
    "Instead, we get \"abc!def!ghi!\" . The + quantifier greedily eats \n",
    "up as much of the string as it can and only stops at the last \n",
    "exclamation point, not the first.\n",
    "        \n",
    "We can make + as well as * into non-greedy quantifiers by \n",
    "putting a question mark after them.This version says, “Give me one or more wildcard characters, but only as many as you\n",
    "see up to the first exclamation point, which should also be included.” Sure enough,\n",
    "this time we get \"abc!\" .\n",
    "        \n",
    "This version says, “Give me one or more wildcard characters, but \n",
    "only as many as you see up to the first exclamation point, which \n",
    "should also be included.” Sure enough, this time we get \"abc!\" .\n",
    "READ\n",
    "    \n",
    "puts /.+?!/.match(string)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# regex anchors & assertions (11.5.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# lookahead assertions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# lookbehind assertions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# conditional matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# modifiers (11.5.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting strings to/from regexes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# string-to-regexp idioms (11.6.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# regex to string (11.6.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Common uses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\"1\", \"2\", \"3\", \"4\", \"5\", \"6\"]\n",
      "[[\"Leopold\", \"Auer\"], [\"Jascha\", \"Heifetz\"]]\n",
      "Auer's first name was Leopold.\n",
      "Heifetz's first name was Jascha.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"Leopold Auer was the teacher of Jascha Heifetz.\""
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# string#scan (11.7.1)\n",
    "<<READ\n",
    "scan goes from left to right, testing repeatedly for a match.\n",
    "the results are returned in an array.\n",
    "READ\n",
    "\n",
    "# harvest all the digits in a string:\n",
    "puts \"testing 1 2 3 testing 4 5 6\".scan(/\\d/)\n",
    "\n",
    "# parenthetical groups\n",
    "str = \"Leopold Auer was the teacher of Jascha Heifetz.\"\n",
    "puts violinists = str.scan(/([A-Z]\\w+)\\s+([A-Z]\\w+)/)\n",
    "    \n",
    "# scan with a code block\n",
    "str.scan(/([A-Z]\\w+)\\s+([A-Z]\\w+)/) do |fname, lname|\n",
    "    puts \"#{lname}'s first name was #{fname}.\"\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# string#split (11.7.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sub/sub!, gsub/gsub! (11.7.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# case equality and grep (11.7.4)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Ruby 2.7.1",
   "language": "ruby",
   "name": "ruby"
  },
  "language_info": {
   "file_extension": ".rb",
   "mimetype": "application/x-ruby",
   "name": "ruby",
   "version": "2.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}