Skip to content

Commit

Permalink
A few corrections and improvements to the model
Browse files Browse the repository at this point in the history
  • Loading branch information
palewire committed Aug 12, 2022
1 parent 8f61f75 commit b4ad1ad
Show file tree
Hide file tree
Showing 9 changed files with 370 additions and 178 deletions.
52 changes: 33 additions & 19 deletions _notebooks/input/labeled.csv

Large diffs are not rendered by default.

241 changes: 140 additions & 101 deletions _notebooks/label.ipynb

Large diffs are not rendered by default.

Binary file modified _notebooks/output/path-and-text-model.pickle
Binary file not shown.
Binary file modified _notebooks/output/path-only-model.pickle
Binary file not shown.
127 changes: 127 additions & 0 deletions _notebooks/output/wrong.csv
@@ -0,0 +1,127 @@
text,url,is_story,prediction,is_accurate
20 Under 40,https://www.lagrangenews.com/20-under-40/,True,False,False
Bela’s Pilgrim,https://jewishcurrents.org/belas-pilgrim,True,False,False
Central/Eastern Europe,https://100r.org/about/the-reporters/central-eastern-europe-staff/,False,True,False
Trouble in Kenya's Flower Fields,https://100r.org/2017/12/trouble-in/,True,False,False
Follow the storms as they come in.,https://www.11alive.com/radar,False,True,False
Changing Child Care,https://19thnews.org/collections/changing-child-care/,False,True,False
SPONSORED: Options for breaking free from excruciating facial pain,https://6abc.com/trigeminal-neuralgia-facial-pain-blood-vessel-temple-health/12063720/,False,True,False
Press Information,/2020/09/press-information-2647452975/,False,True,False
Midterm elections,/hub/2022-midterm-elections?utm_source=apnewsnav&utm_medium=sections,False,True,False
"byJonathan Stoltman, Ashton Marra & Kristen Uppercue",https://www.100daysinappalachia.com/author/jonathanashtonkristen/,False,True,False
Read More >,https://ashland.news/reader-photo-navigating-a-cloudy-surface/,True,False,False
Tracking the Curve,https://aspenjournalism.org/aspen-area-virus-information-sources/,False,True,False
The complaint,https://avlwatchdog.org/wp-content/uploads/2022/06/Shroat-complaint.pdf,False,True,False
Food & Drink,https://www.baltimoremagazine.com/section/fooddrink/baltimore-regions-best-crab-houses-crab-cakes-soft-shells/,False,True,False
Achieving equilibrium,/sponsored/achieving-equilibrium/article_d7295032-f6f6-11ec-9477-f7533a9c28fe.html,True,False,False
Jack's Cannabis,/marketplace/jack-s-cannabis/business_4000204783.html,False,True,False
"How to remove a Trojan, Virus, Worm, or other Malware",/tutorials/how-to-remove-a-trojan-virus-worm-or-malware/,False,True,False
A Beautiful Resistance,/metro/special-reports/a-beautiful-resistance/?p1=BGMenu_Sections,True,False,False
Heath Alan Mosley,https://breckenridgetexan.com/2022/07/24/heath-alan-mosley/,False,True,False
Josephine A. Garcia,https://breckenridgetexan.com/2022/06/27/josephine-a-garcia/,False,True,False
Comments,/news/local_news/open-records-request-denied/article_ee2e489e-53dc-59bc-855e-454e109c7a41.html#comments,False,True,False
Newell Brands: Q2 Earnings Snapshot,/ap/business/newell-brands-q2-earnings-snapshot/article_644d356f-0cce-5b2c-9526-c4d231277ddd.html,False,True,False
"Tue

9",/calendar/day/when/2022-08-09/,False,True,False
Capitol News Illinois,https://soundcloud.com/user-592090129-621623053,False,True,False
Best of Chicago: Sports & Recreation,https://chicagoreader.com/best-of-chicago/2021-sports-recreation-index/,False,True,False
Read More →,/english/berwyn-mayor-lovero-covid-community,True,False,False
Cory Lum/Civil Beat,https://www.civilbeat.org/beat/uh-gets-700k-to-study-racism-and-lung-cancer/,True,False,False
"Nonprofit News Powered By

Daintry Bartoldus

Miriam Beams

Elizabeth Blalock

Deborah Fleming

George Hudes

Readers Like You",/donate,False,True,False
2022 Elections | Clayton County Board of Commissioners,https://claytoncrescent.org/category/elections/2022-clayton-county-board-of-commissioners/,True,False,False
Explore all Climate Matters materials in our resource library,/resources?type=Climate+Matters,False,True,False
The Howard Center for Investigative Journalism,https://cnsmaryland.org/author/howard-center-for-investigative-journalism/,False,True,False
▶️ Get Outside: Paintball,https://centraloregondaily.com/%e2%96%b6%ef%b8%8f-get-outside-paintball/,True,False,False
More Episodes,https://centraloregondaily.com/trailhead-day-use-theft-suspects-walmart/,True,False,False
2020 National House Vote Tracker,/2020-house-vote-tracker,False,True,False
The 2022 State Legislative Sessions Calendar,https://info.cq.com/resources/2022-state-legislative-sessions-calendar/,False,True,False
Best of the Best 2022,https://lcsun-news.secondstreetapp.com/2022-Best-of-the-Mesilla-Valley/,False,True,False
NewArena,https://newarena.com/entertainment/25-most-overrated-singers-of-all-time/?utm_content=73535&utm_campaign=tb_zs_overrated_1_edg_2_dt&utm_source=tb_ae&utm_term=gannettcompany-lascrucessun-news&utm_content=25+Most+Overrated+Singers+of+All-Time,False,True,False
"Admissions Officer£28,396-£33,682pa (NHC29-35)",https://www.fish4.co.uk/job/13234446/admissions-officer/?TrackID=1270958&utm_source=tm-bedfordshirelive&utm_medium=referral&utm_campaign=widget,False,True,False
"COURSE LEADER – ENGINEERING£30,840 - £37,903 per annum",https://www.fish4.co.uk/job/13308628/course-leader-engineering/?TrackID=1270958&utm_source=tm-bedfordshirelive&utm_medium=referral&utm_campaign=widget,False,True,False
"Good Job, Everyone. Great Work.",https://defector.com/good-job-everyone-great-work-kyler-murray/,True,False,False
GET OUR BOOK,https://www.amazon.com/Desert-Oracle-Strange-American-Southwest/dp/0374139687/,False,True,False
10 Years of the Higgs Boson,//www.discovery.com/space/higgs-boson,True,False,False
Naked and Afraid XL,//www.discovery.com/shows/naked-and-afraid-xl,True,False,False
Tracy Morgan's Best Shark Week Quotes,//www.discovery.com/shark-week/sharks-with-tracy-morgan,False,True,False
The Best Men's Shoes for Walking and Standing All DayThese shoes were made for walking and to provide optimal comfort.trabladzer|,https://www.trabladzer.com/products/mens-good-arch-support-outdoor-breathable-light-travel-sneakers1?st=&utm_source=taboola_tl&utm_medium=referral&tblci=GiATAsDr9w8K2YTz1I4NsIoVczMOWm4TLTZGJJNO-4R1FCDQyVkomPSD-c2cn4vtAQ#tblciGiATAsDr9w8K2YTz1I4NsIoVczMOWm4TLTZGJJNO-4R1FCDQyVkomPSD-c2cn4vtAQ,False,True,False
"If you really want change, you need to put in work to make it happen",/comment/climate-change-tory-party-put-work-make-happen-martin-lewis-b1015510.html,False,True,False
Submit a letter to the editor,https://evanstonroundtable.com/submit-a-letter-to-the-editor/,False,True,False
Your California Privacy Rights,https://privacy.thewaltdisneycompany.com/en/current-privacy-policy/your-california-privacy-rights/,False,True,False
Donors – 2011-2012,https://www.floridabulldog.org/donors-to-broward-bulldog/donors-2011-2012/,False,True,False
"July 25, 2022 5:00 am",https://www.floridabulldog.org/2022/07/,False,True,False
25 Best Large Workplaces in the Bay Area,https://fortune.com/best-workplaces-bay-area/2022/,False,True,False
Pelosi Must Go,https://freebeacon.com/columns/pelosi-must-go/,True,False,False
Advertising Terms and Conditions,https://cm.gainesville.com/advertising-terms-conditions,False,True,False
"Website by CMIT Solutions of Oak Park, Hinsdale and Oak Brook",https://cmitsolutions.com/oak-park-hinsdale-oak-brook/,False,True,False
April 2022,https://harpers.org/archive/2022/04/,False,True,False
August 2022,https://harpers.org/archive/2022/08/,False,True,False
65% off your 1st box + 30% off in month 1 - Gousto discount code,https://www.independent.co.uk/vouchercodes/gousto,False,True,False
"8

Bad Education",https://jewishcurrents.org/bad-education,True,False,False
Click: Your latest updates from the News Letter,https://www.newsletter.co.uk/news/latest,False,True,False
Just Married in New Jersey,/just-married-in-new-jersey-41505296,False,True,False
"Sun, Jul 31 @9:00amPride Month Craft Grab & GoMarion Branch",https://newjersey.news12.com/new-jersey-events#!/details/PRIDE-MONTH-CRAFT-GRAB-GO/9939967/2022-07-31T09,False,True,False
Journalism of tolerance,https://www.kyivpost.com/special-project/journalism-of-tolerance,True,False,False
सबै,https://www.nayapatrikadaily.com/news-article/7/MultiMedia,True,False,False
टर्निङ प्वाइन्ट्स-२०२०,https://www.nayapatrikadaily.com/news-article/28/TurningPoints,True,False,False
"Alia Crum says that our perceptions are always filtered through our mindsets — and these mindsets shape our lives in profound ways. In the second of two episodes, the psychologist explains how our beliefs about food and exercise affect our bodies.",/radio,False,True,False
Deutschland,/nachrichten/deutschland/deutschland-nachrichtenfeed-100.html,False,True,False
Best Colleges,https://www.indiatoday.in/bestcolleges/2022,False,True,False
"Sun, Jul 31 @2:00pmDiamond Dan: Neil Diamond TributeSoaring Wings Vineyard & Brewing",https://metro.newschannelnebraska.com/calendar/#!/details/DIAMOND-DAN-NEIL-DIAMOND-TRIBUTE/10579958/2022-07-31T14,False,True,False
"Tue, Aug 02 @7:30pmThe IguanasThe Waiting Room Lounge",https://metro.newschannelnebraska.com/calendar/#!/details/THE-IGUANAS/10514758/2022-08-02T19,False,True,False
Update My Sustaining Membership,https://donate.nprstations.org/kvpr/update-my-sustaining-membership,False,True,False
Consumer Reports: Best dehumidifiers,https://www.nbcnews.com/select/shopping/consumer-reports-dehumidifiers-ncna1289320,False,True,False
Contact,https://www.nbcnews.com/news/us-news/nbc-news-digital-editors-n893846,False,True,False
Toronto Real Estate Prices Might Be Lower Than You ThinkToronto Real Estate | Search Ads|,https://compare-intl-real-estate-prices-toronto.zone?ref=taboola-gannettcompany-reporternews&sub_id=gannettcompany-reporternews&sub_id2=350-realestatetoronto-taboola-all&tbid=1353010&tbclick=sysclick&compkey=Toronto+Real+Estate+Prices&rskey=Toronto+Real+Estate+Prices+Might+Be+Lower+Than+You+Think&tblci=GiD9o_6V7VrYXJIlgI8CtTLHmAZreecDqvT4ki87jxdVhCCillQo8LStrbX6hsWaAQ#tblciGiD9o_6V7VrYXJIlgI8CtTLHmAZreecDqvT4ki87jxdVhCCillQo8LStrbX6hsWaAQ,True,False,False
About Us,https://www.postcrescent.com/story/news/local/2021/10/03/post-crescent-contact-information-staff-directory/8308687002/,False,True,False
Use of force: a double standard,https://southsideweekly.com/use-of-force-a-double-standard/,True,False,False
Book Review: ‘The Third Coast’,https://southsideweekly.com/book-review-the-third-coast/,True,False,False
"Braves take down Angels, 8-2",https://www.wjbf.com/sports/braves-take-down-angels-8-2/,True,False,False
Saved articles, https://www.startribune.com/subscriber/saved-articles,False,True,False
img.full-width1{flex-shrink:0;object-fit:contain;min-height:50%;height:auto;width:100%},/bal-ac-dan-belson-staff.html#nt=pf-double chain~homepage~flex feature~curated~river-four~EMHZ5IVNIVG7JAUIO6CX5WQAAM~1~1~4~7~art yes,False,True,False
img.full-width1{flex-shrink:0;object-fit:contain;min-height:50%;height:auto;width:100%},/bal-ac-dan-belson-staff.html#nt=pf-triple chain~home-double-rec2~recommender~automated~politics-rec~FP6CDJS2ONHMFAI7D4S4YC6FF4~1~6~1~3~art yes,False,True,False
img.full-width1{flex-shrink:0;object-fit:contain;min-height:50%;height:auto;width:100%},/ac-cn-winegrad-bio-staff.html#nt=pf-double chain~homepage-duplicate~flex feature~curated~box-9~URBBGCGFM5GMPBDMCTJLYCJTQE~1~1~8~12~art yes,False,True,False
Credit cards,https://www.ft.com/stream/f769d21e-0045-465a-9e7c-418b2de754b0,False,True,False
Main Events 07/29/2022,/entertainment/main-events-07-29-2022/article_383fa845-afb6-5bab-a23a-44efeaf8d330.html,False,True,False
Carmen Lomana,https://www.larazon.es/gente/famosos/20220730/obrfiob6incxbe4dtwo7lqn6uu.html,False,True,False
Sign up for our email,/info/2018/sep/17/guardian-us-morning-briefing-sign-up-to-stay-informed,False,True,False
"switch to the
International edition",https://www.theguardian.com/preference/edition/int,False,True,False
Explore nearly two decades of Cook County court data,https://charges.thecircuit.cc/en/,False,True,False
Access Toolbox →,https://go.nationaljournal.com/PAC-Manager-Toolbox-2022,False,True,False
Read More ...,https://www.okayplayer.com/music/flying-lotus-yasuke-netflix-interview.html,True,False,False
Los Angeles Times for Students and Educators,https://www.latimes.com/about/marketing/los-angeles-times-for-students-and-educators,False,True,False
The Best Beach Umbrellas for All Day Outdoor Fun,https://www.latimes.com/bestcovery/beach-umbrella,False,True,False
Enjoy Your Time Outdoors With the Best Cooler Backpack,https://www.latimes.com/bestcovery/cooler-backpack,False,True,False
Enjoy Your Time Outdoors With the Best Cooler Backpack,https://www.latimes.com/bestcovery/cooler-backpack,False,True,False
Clarity,https://www.latimes.com/projects/clarity,True,False,False
9 little-known ways to see the Hollywood sign,https://www.latimes.com/travel/list/hollywood-sign-viewing-hikes,False,True,False
Alpha Athletics Alpha One Golf Glove Review,https://www.latimes.com/bestcovery/alpha-athletics-alpha-one-golf-glove-review,False,True,False
Manual vs. Electric Knife Sharpeners: Which Should You Choose?,https://www.latimes.com/bestcovery/manual-vs-electric-knife-sharpeners-which-should-you-choose,False,True,False
Contact Us,https://help.nytimes.com/hc/en-us/articles/115015385887-Contact-Us,False,True,False
Letter BoxedCreate words using letters around the square.,https://www.nytimes.com/puzzles/letter-boxed,False,True,False
Spelling BeeHow many words can you make with 7 letters?,https://www.nytimes.com/puzzles/spelling-bee,False,True,False
© 2022 The New York Times Company,https://help.nytimes.com/hc/en-us/articles/115014792127-Copyright-notice,False,True,False
VertexConnect the dots to reveal the hidden picture.,https://www.nytimes.com/puzzles/vertex,False,True,False
"New York Times GamesSubscribe for full access to Spelling Bee, Wordle, The Crossword and more.",https://www.nytimes.com/subscription/games,False,True,False
Eiko OjalaDo You Have a Question for Our Health Editors?,https://www.nytimes.com/interactive/2021/well/ask-well.html,False,True,False
US Elections,/election/2022,False,True,False
Content by FinanceBuzz7 ways to avoid financial stress and earn cash,https://financebuzz.com/financial-stress-lz2k?utm_source=cnn&utm_medium=paid-native-rs&utm_term=ros&utm_campaign=financial-stress&utm_content=text-only&adgroupid=cnn-nb-financialstress,False,True,False
Why McConnell can't give up supporting Trump,https://plus.cnn.com/plus/title-2366485,True,False,False
CNN+ Mark Meadows' texts revealed,https://plus.cnn.com/plus/title-2366485,True,False,False
cnn+ Declassified: Untold Stories of American Spies,https://plus.cnn.com/plus/title-2081265,True,False,False

0 comments on commit b4ad1ad

Please sign in to comment.