机器学习实战第十二章——条件随机场（CRF）.ipynb

{
 "cells": [
  {
   "attachments": {
    "%E7%BA%BF%E6%80%A7%E9%93%BE%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA1.jpg": {
     "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAYEBQYFBAYGBQYHBwYIChAKCgkJChQODwwQFxQYGBcUFhYaHSUfGhsjHBYWICwgIyYnKSopGR8tMC0oMCUoKSj/2wBDAQcHBwoIChMKChMoGhYaKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCj/wAARCACyAR8DASIAAhEBAxEB/8QAHQABAQACAwEBAQAAAAAAAAAAAAYFBwMECAIJAf/EAEcQAAEDAwEEBQYLBQYHAAAAAAEAAgMEBREGBxIhMRM3QVF1FBUWImGzCCMyNlRxgZGWtdMXM1ZisSQlgpWh0kJDcsHR4fD/xAAYAQEBAQEBAAAAAAAAAAAAAAAAAQIDBP/EACwRAQACAQIDBwQCAwAAAAAAAAABEQIh8BIxQQNRYXGBkbEE0eHxE6EUIsH/2gAMAwEAAhEDEQA/APVKIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIC1PrSaTZlqmfWNJTT1GnLriK80tO3edFUAYiqWN73HEbuWctPErbCldqGnavVeia6z26SCOpnkgc107iGAMmY85IBPJp7OalaxW46r0qXT2aafrKOKu1FqGMN1LfHtnqmZ3vJYwPiqZp7mN597i49ytl1rnTSVlBPTw1lRRSSN3RUU+50kftbvtc3P1gqT2QU7aTRrqdj5ZGRXK4Rh80hke4CsmGXOcSXHvJ4la8uiVUX3z+VqiIoCIiAiIg1rtJo6jS97p9oNmhfL5LGKe90sY41NFnPSAdskXyh3t3hnC5tnNBPf7xVa9vMD4p6+LoLTTSjjSUIOWkjsfKfXd3DdHYVWaytc170he7VSOjZUV1DNTRukJDQ57C0EkAnGT2Arp+bYrfs6jtl4rzRQ01sbT1VXTy9H0QbGGve15HDkSDhSP8AWJ8OXrd78ZWY4piO/n6VW/CFIiiNnFogoqi5V9ptLLJYqxsLaO3shEJduB29UPiGNxzw5owfW3Y272D6ot1qYpmJsREUUREQEREGC1vpmk1dpqrtFa58XSgPhqI+ElPK05ZKw9jmuAP+natW0ddfNo8lPofUNFLSOtMgOp52gtjqgwgxRxHtbNgPPLDQR2hbvUrprTtXa9bavvFRJA6mvElK6BrHEvaIodx2+CABx5YJ4dyRz15c/XpvrUWTrG/XfToqI2NjjayNrWMaA1rWjAAHYAvpS2jqh8991kx7Yg2G6tjaWRNYSPJac+sQAXHJPF2TjAzgACpV7p76n31KrQREUBERAREQFpm811dslv8AXQ2q2T3GxahkdJbKSEZ8mubucP8ALFJ8vuaQ7gtzKV15p2r1BNpl9FJAwWy8QXCbpXEb0bGvBDcA5d6w54HtUrWPb0nn9/OFvSff1jl78vV9bO9LHS1idHVzCqvFbK6suVX2z1D+LiP5Rwa0dgA9qqFJa7qriyu0zb7Xcprb5xuDqeeeGKN7wwU80mG9I1zQcsbxIK7Oz241dz00JLjOamphq6qkdO5rWulEM8kYc4NAaCQwE4AGc4AWuaVw1fX8/aVIiIoCIiDhq6qGjgdNUyCONvMn+g7z7Fg6S92+kaYqW3VFPTl7pCY4WtaXOcXOdug5ySSTwySSV1tRSOmvQicfi4Imua3+Zxdk/cAPvXSXTHC4uXl7Xt5xyqFnS1ENXA2ankbJG7k4LlUNS3uCw1rjUx10lPUsJLKOimq3B7SOO5ExzgCDxcRjgBniF3/Ty0fQ9Sfhy4/oLGUVNO/Z58eNqpFK+nlo+h6k/Dlx/QT08tH0PUn4cuP6CjaqRSvp5aPoepPw5cf0E9PLR9D1J+HLj+ggql1LvDQ1FtqIbtFTzUD2Fs0dQwPje3uc08CPYsB6eWj6HqT8OXH9BY+s1DTX+tjhpIrlFDTt6Vza23VFJvOJw0gTMbvYweWcZGexWI4tGc8+DHidiyHTWn3zej2nIbeybHSyUNDFAJMZxvAYccZOMjtKq6KrgrYBLTSB7M4PYQe4g8QfYVHLtWOV0F8iaz5NQxzHjvLRkH7MEfaumWGlvN2fbzOVSrkXSvdRWUlorKi10PnCuiic+Ck6ZsPTPA4M33cG55ZPJcHltz862+n80/2KanfJU1XlLf7NKN3dj3Ob85d6w4Dd9q5b3v5etlEWPsdXcKyklkutt82zNnkjZF07Zt+NriGSZbwG83Dt3mM4Kwvp5aPoepPw5cf0EFUilfTy0fQ9Sfhy4/oJ6eWj6HqT8OXH9BBVIpX08tH0PUn4cuP6C61w11b30ckdHTX+OokxHG+aw10TGuccAl74Q1oGc5JARJmtWYmultt1VUtpaYyVEsm/UGmjaC5+6G5e44BdhrRxOcABdm3XmlrpRE3pIZyMiOVuCfqIyD9hUvFG2KNrGDDQML4qQehc5jt2RnrscObXDiCuv8cU8kfUzfgvUXFRzeUUkExGDIxr8d2RldS6Vdwpqu2x0Ft8shnn6Oql6dsfkse649Jg8X+sGt3Rx9bPYuT2eLIIsZRVlymqbqyqtXk8NPIG0cnlDX+WN3Ad7A/d+sS3B7s9q5bFU1tZZ6Soutv8218sYdNR9M2boXdrd9vB2O8Jve9B3kXFV1MFHSzVVZNFBTQMdJLLK8NZGxoyXOJ4AAAkkqa/aPof+MtN/wCaQf7kFUilf2j6H/jLTf8AmkH+5P2j6H/jLTf+aQf7kGau81vpzTT3COJ8sTy+nzHvva/dLSWcMg7riMjsJ7CsdQ3u20jTFFQy0MBe6QkQtDN5zi5ziGE8SSSSRzJJWDjulJfKie5W+sp62ke8xQT08rZIyxpwd1wJB9YO5f8AZcq6xho8mf1ExlULdj2yMa9jg5jhkOByCO9f1T+kpXBlXS/8uJ4cz+UOGSPvBP2qgXOYqaenDLiiJERFGmD1DbZZ5GVlI3fmY3cfHnBe3ORj2gk/eVNuq2Nq46QsmFZIxz2U5icJHNaWhzg3HEAuaCeQ3h3hbAUrcutPT3gtz9/QLcZzGjjn2GOc2yGnrbLTukqqtoZPI0MbHnPRt54J7yef1BZpEWZm3THGMYqBERRoREQFiNQW6SrZFPSgGphyA0nG+04y3PfwBH/tZdFYmkyxjKKlAT1TKeaGGpZNFPO8xxRPicHSPDS7daMesd1rjwzwaTyCoNP22ZlQa2sZ0bt0siiPNoPMn2nA4dg+tdTWXzj0J41J+XViqlqc5nRyw7DHCbERFh2EREBERAXFV07Kullp5QejlaWOxzwVyogiainqqFxjq4ZHAcBNHGXNeO/hndPsP+q+LXTuv0EMlHk26dgf5UODZIyMgsP/ABZB4EcMHOVcqV2T9VmjfBaL3DFv+SXn/wAfG7VLWhrQ1oAAGAB2L+oiw9AiIgIiICIiCavltniq5KulidNFLgyxsGXNcBjeA7QQBwHH68rCxVInrH0dNHLLWMY2R8AYWuY1xcGudnG6CWuAJ57p7ir9Stt609Q+C2z39etxnMRThl2GOU2zNjt7qClcJS11RK7fkLeQOMAD2AAf17VkURYmbdoiIioEREUUrcutPT3gtz9/QKqUrcutPT3gtz9/QIKpERAREQEREBERBK6y+cehPGpPy6sVUpXWXzj0J41J+XViqkBERAREQEREBERAUrsn6rNG+C0XuGKqUrsn6rNG+C0XuGIKpERAREQEREBERAUrbetPUPgts9/XqqUrbetPUPgts9/XoKpERAREQFK3LrT094Lc/f0CqlK3LrT094Lc/f0CCqREQEREBERAREQSusvnHoTxqT8urFVKV1l849CeNSfl1YqpAREQEREBERAREQFK7J+qzRvgtF7hiqlK7J+qzRvgtF7hiCqREQEREBERAREQFK23rT1D4LbPf16qlK23rT1D4LbPf16CqREQEREBSty609PeC3P39AqpSty609PeC3P39AgqkREBERAREQEREErrL5x6E8ak/LqxVSldZfOPQnjUn5dWKqQEREBERAREQEREBSuyfqs0b4LRe4YqpSuyfqs0b4LRe4YgqkREBERAREQEREBStt609Q+C2z39eqpStt609Q+C2z39egqkREBERAUrcutPT3gtz9/QKqUrcutPT3gtz9/QIKpERAREQEREBERBK6y+cehPGpPy6sVUpXWXzj0J41J+XViqkBERAREQEREBERAUrsn6rNG+C0XuGKqUrsn6rNG+C0XuGIKpERAREQEREBERAUrbetPUPgts9/XqqUrbetPUPgts9/XoKpERAREQav277Uotm1gidTxNqLvV5bTxOPBo7Xn2BeTDt01w7UsF7fcInVUEckUbDH8W2OQsc9oGeRMbD/hC2n8Nmz1jq2wXdjHuoWRvge4cmOJBH9Of1Ly2qnN+hew3aZDtJ00+pfE2nudK4R1MIORnscPYVsleWPgS2athbqC8SxubRTtZBGSMbzmkkkd444+xep1CBERFEREBQe2XaHS7ONKOuU0fT1czuipYM43347T3DmrxebfhqWesqtM2a5wNe+mpZy2YN5M3hwcft4JCS0fdNuuuLleqO4zXCNrqOc1FPC2PEcbyx0ecZ4+pI8f4ivU/wf8Aay3aRap4K+JkF6owOmaz5MjTyePr7v8Axk+Bl6Q+BbZayXVl0u4a9tDBB0Jdj1XvPZ9Y5/aqTo9iIiKKIiICIiAuKqnjpaaWonduxRML3nuAGSuVYXW1BNdNI3ehpnFs09M9jCOecdiJPJ5F2l/CP1HcrzUQaTmbbrXE8tjla3MkmD8rPYD3f05LJ7BNvVdQ19q0xqVsT7Tux0dLMxpDoAAGsae8ch93flecqymmoquamqo3RTwvLJGO5tcDghZPRtrqrzqq1UFvY99RNUsDdwZI9YEn7BxVStH6dgggEHIKLgoYXU9DTwvdvOjjawnvIGFzqNCIiAiIgIiINEfCI21TaDmisun445LzKwSPkkGWwtPLh2k//cuPnnT23jWdq1JLeJ6uOuknZHDOyZvB8THPc1o7sGSTB/mKynwtbPWUG1Satna801bC18MhHDhwLR9XD71pRVmIt+l2zjV9FrnSVFe7eC1kww+M82PHNv3qmWkvgkWastey5s9a17BWzumiY8YwzsP281u1RYERQ161lPpnXcFv1KaOm05c4v7vuGCzo6hgy+GZxcW+sPWa71eRHE8Utelqm/Wa33+2TW+8UsVXRyjDo5G5BWom/Bp0GLh5R0NaYc56Aznd+9bB2f6huOqoK27yU0VNYZpALSHMcJ54gMGZ+TgNeeLRgHdGTnPCsV1hKiXSstqobJbYaC1UsVLSQjdZFG3AAXdRFFEREBERAXWudBSXSgnorhBHUUszSySKQZDgexSmttV1mkb9Z6m4x03olWO8kqqoMd0tHUOPxb3uzjonfJPAbpIOcHC5NJaoq9U6gukttip/RWjzSw1ZBL6ypa74x0ZDsdE3G7nB3nZwcDijXlutx7k6c9719kJVfBr0HPcTUtgrIoid4wMnIb9nd9i2tpnT9s0zaYbbZKSOlo4hwYwYye895WVREoRERRERAREQEWM1O67ssFc/TYpHXhkZdTMq2udE9447rt1zSM8s54E544woGLaj6QWyy0ukaeKTVFykMctFVNJFt6NwFQ+oaC1wDDkAZBcSMJGs1G99e7qTpFzvf9uXX2xPR+tK91fcKN9PXv8AlzUztwv9pA5ld/Z5sn0roOV1RZaImtcN01Mzt94HcCeSvRnAzzREqBERFEREBERARFrup2ht0zqe92rXMlJQ08cLrha6yNrmsqqZo9eM5JzMw8CB8oOBDQpa1ak1ro6ya0tRt+oaJlTBnLTycw97T2Fa5svwcdCWy5Nq301TVhjt5kU8pcwewjtH1rYuhbjebvYm3G/0cVBJVSOlpqRrSJIac/uxKS4gyEcTjAGcY4FUK1NxpLMVOsPiCKOCFkULGxxMAa1rRgAdy+0RRRaj2iUf7VbvVaGopnw2O3lst4uETWkifGYqePeBG8Dh7yOQAbkErbiKTF81ia5NdbPNUXI2O42O8W4zar08GwTUdKY4/K48fFTRbzmtDXt9owQRw4BUWz+9V9/02K67U9PTVoqqmnkhp3l7GdFO+MAOPF3Bg44GTxwOSo1iNLWXzDbJKPp/KN+rqarf3NzHTTPl3cZPLfxntxnhyWrvWee/n9M1ERUd/wDWvwy6IiiiIiAiIg1rtNqpNVV42eWh3xtdCJbxVNAIoqMniBkEdJJgtaOJAy7HAFdDQFwuekae5aBnpI6q62mmdPYyXNhbcqXiGZdyD2u9V/DuODnK2yujfaasrbNW01srhb62aJzIavoul6FxGA/dJGcc8ZU5RPjuPb79+l5zG/P3+3cwuk7rdZ7zdrRep7fWVFAyCR1TQQPgjDpA49E5jnvO80Na7O9xbI3gO2oU3oew1+nbc6irKu21EA9aM0dDJTvLySXvkc+aUyOcSCXHBzkknPCkWpZgREUUREQEREGE1pqWi0lp6pu1w33sjwyKCMZknlccMiYO1ziQB9/IFagpLZe9nF7btCvoZMy+vDdSU8bGkW5rnDonxkDJbHkNfxOeLuPMb6RI0ni6/wDOsev6onWOHd9Pb9sNp+8tutbeoo5GyMoattO0tj3RgwxycHb7t/8AeZ3sN7scN45lYix2XzXcb5VdP0vnOsFXu7m70WIY4t3OTn93nPDnjsysur0jyj41BERQEREBERAWktYWmq2x3Kp8z1LaSzacmcbfWGNr21tyZ/1A5hZgtOODiTz3Vu1FK1vd9/p8+S3o11YNpjKywW99baa1+on1jrXVWqlEZkhq2Mc9zcve1oYWsLg4uxgjjzVnpy8wX60xV9NFNC1z3xvhmDQ+KRjyx7HbpIyHNI4EjhwJXFe7L5zudirOn6LzXVuqtzc3ulzDJFu5yMfvM548sduU0tZfMNsko+n8o36upqt/c3MdNM+Xdxk8t/Ge3GeHJau9UqqrfP8ADLoiKAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiD//2Q=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**摘要**  \n",
    "\n",
    "1. 马尔科夫随机场\n",
    "2. 条件随机场\n",
    "3. 线性链条件随机场 \n",
    "\n",
    "**马尔科夫随机场**\n",
    "\n",
    "上一次我们学习了隐马尔可夫模型，它属于有向图模型。 \n",
    "\n",
    "我们设无向图$G(V,E)$中节点的联合概率分布为$P(Y)$，$Y$是随机变量。  \n",
    "\n",
    "如果存在一个无向图，如下所示：  \n",
    "\n",
    "\n",
    "<img src=\"https://pic4.zhimg.com/80/v2-1a8572140431c85513ac56099c55cc0e_hd.jpg\",width=400,height=400>\n",
    "\n",
    "\n",
    "其中，A、B是无向图中被节点C分开的任意节点的组合。节点集合A、B、C所对应的随机变量分别为$Y_A$、 $Y_B$ 、 $Y_C$ 。如果满足下列马尔可夫条件独立：\n",
    "$$\n",
    "P(Y_A,Y_B|Y_C)=P(Y_A|Y_C)P(Y_B|Y_C)\n",
    "$$\n",
    "那么我们称此联合概率为马尔可夫随机场。\n",
    "\n",
    "\n",
    "\n",
    "## 条件随机场\n",
    "\n",
    "条件随机场（conditional random field，CRF）：给定随机变量X的条件下Y的条件概率分布的马尔可夫随机场。\n",
    "\n",
    "这句话怎么理解呢？\n",
    "\n",
    "在 $P(Y|X)$ 中，$Y$是输出变量，表示标记序列，$X$是输入变量，表示需要标注的观测序列。通俗讲，这里的$（X，Y）$就是我们已知的训练样本。\n",
    "\n",
    "$P(Y|X)$ 满足马尔可夫随机场，即满足下面的公式：\n",
    "$$\n",
    "P(Y_v|X,Y_w,w\\ne v)=P(Y_v|X,Y_w,w\\sim v)\n",
    "$$\n",
    "式子中 $w\\sim v$ 表示在图$G(V,E)$中与节点v有边链接的所有边， $w\\ne v $表示节点v以外的所有节点。上面的公式表明 $Y_v$ 只与直接连着的节点有关。\n",
    "\n",
    "\n",
    "\n",
    "## 线性链条件随机场\n",
    "\n",
    "下面我们主要解释应用比较多的线性链条件随机场。其图结构如下所示：\n",
    "\n",
    "![img](attachment:%E7%BA%BF%E6%80%A7%E9%93%BE%E6%9D%A1%E4%BB%B6%E9%9A%8F%E6%9C%BA%E5%9C%BA1.jpg)\n",
    "\n",
    "$Y_i$ 发生的概率只与直接相连的 $\\left\\{ X_i,Y_{i+1},Y_{i-1} \\right\\} $有关。线性链条件随机场主要用在标注问题中，$X$表示输入观测序列，$Y$表示对应的输出标记序列或状态序列。\n",
    "\n",
    "对于线性链条件随机场的理解有一篇文章写得非常好，我直接贴上地址，我也是通过这篇文章从而理解了这个模型，所以我在这里不赘述，直接移步原文：\n",
    "\n",
    "[Information to Conditional Random Fields](http://blog.echen.me/2012/01/03/introduction-to-conditional-random-fields/)\n",
    "\n",
    "中文翻译版：\n",
    "\n",
    "\n",
    "[CRF介绍](https://www.jianshu.com/p/55755fc649b1)\n",
    "\n",
    "\n",
    "以上就是条件随机场的全部内容，希望大家能有所得，下一次我们一起学习最大熵模型。\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}