tipuesearch_content.json

{"pages":[{"text":"I am a computer programmer from Pakistan . I have made a lot open source contributions. I have contributed mainly to youtube-dl . It's a python script which allows you to download videos from various websites like youtube, facebook, dailymotion, metacafe, etc . I also run a website with the name of soundcloud-dl It allows you to search and download music from soundcloud.com . I know that sounds cool :D . I have done all of it's backend programming in python. I also have some useful scripts on my github page and if you would like to check them out then you are most welcome. This blog will serve you new python tips and tutorials every other day. This blog is for everyone who wants to enhance his knowledge about this language. Lastly if you have any comments, suggestions or want my help with something then feel free to email me .","tags":"pages","loc":"http://yasoob.github.io/blog/about","title":"About"},{"text":"Beginner's Delight Style Guide and Idioms Dictionary Decorators Generators Iterators Yield Context Managers Unicode Networking Metaclasses Documentation Sphinx Debugging Logging Testing Environments and Environment Management Profiling Packaging Deployment Fabric Warts and Gotchas Web Frameworks Flask Web2Py Django Bottle Tornado API and Web Services Scraping Mobile Development Kivy Google Glass Resources Libraries GUI Programming Android SL4A WSGI Databases SQLAlchemy Data Mining Data Analysis Pandas Design Patterns Concurrency Patterns Concurrency and Distributed Systems Python 2 vs Python 3 Porting to Python 3 Books Free Paid Online Courses and Challenges Discussions Conferences and Events Videos Editors and IDE's for Python programming Big Data Curated Python resources from other websites Newsletters Miscellaneous Beginner's Delight Beginner's guide to Python The Hitchhiker's guide to Python Learn Python the hard way Learn Python Google's Python class Python tracks at Codecademy Python Monk Python for Beginners Best way to learn python Python Tips, tools and resources Python Koans: Learn Python through TDD LearnStreet Python course Beginner projects for a python newbie Learn X in Y minutes - Python Python Practice book Facts and myths about Python names and values args and * kwargs in python explained Common newbie mistakes in Python- Part 1 Common newbie mistakes in Python- Part 2 PySchools Some cool Python Tricks if x and if x is not None Better Python APIs Declaring dependencies in Python Storing and Loading Data with JSON - Serialization/deserialization A Guide to Python's Magic Methods Efficient String Concatenation in Python Be Pythonic: init .py Building an open-source Python application the right way Python progression path - From apprentice to guru Tutorialspoint python tutorial Python Importing Python's super() considered super! No Naked Excepts Supercharge Your Python Developers Style Guide and Idioms PEP 8 - Style Guide for Python Code Code Like a Pythonista: Idiomatic Python Dictionary Python hash, id and dictionary order Notes on dictionary implementation Python: The Dictionary Playbook Decorators Understanding Python Decorators in 12 easy steps Common uses of Python decorators - Quora Common uses of Python decorators - Stack Overflow Python Decorators Decorators I: Introduction to Python Decorators Decorators and annotations The dark side of decorators Thinking out aloud: Python decorators Using Python decorators for registering callbacks Meta-matters: Using decorators for better Python programming How can I make a chain of function decorators in Python? The Python Decorator Library Generators Generator Tricks for Systems Programmers Delegating to a Subgenerator in Python 3 Iterators Understanding Iterators and Iterables Yield Python yield keyword explained Context Managers Understanding Python's with statement The Python \"with\" Statement by Example Unicode Explain Unicode like I'm five years old Unicode Pain Python encodings and Unicode The Updated Guide to Unicode on Python Networking An intro to Python socket network programming Metaclasses A primer on python metaclasses What is a metaclass in Python ? Yet Another Python Metaclass tutorial Documentation Documentation is King Make your open source project documentation suck less Sphinx Quick Sphinx documentation for Python Environments and Environment Management Virtualenv - The official documentation Virtual Environments \u2014 The Hitchhiker's Guide to Python Virtualenv Tutorial A Primer on virtualenv How to use Python virtualenv Making Virtualenv play nice with Git Extension to Virtualenv VirtualEnv Burrito = virtualenv + virtualenvwrapper Debugging Debugging Tools in Python Debugging Python Like a Boss Logging Sentry - realtime event logging and aggregation platform Testing Nose Introduction Unittest Introduction How To Use Coverage.py With Unittest An Extended Introduction to the nose Unit Testing Framework Test strategies for your Python projects Profiling A guide to analyzing Python performance Packaging The Hitchhiker's Guide to Packaging Sharing Your Labor of Love: PyPI Quick And Dirty Open Sourcing a Python Project the Right Way 5 tips for packaging your Python projects AOSA: Python Packaging The Package Dependency Blues 5 Simple Rules For Building Great Python Packages Unofficial Windows Binaries for Python Extension Packages Py2app: Standalone Mac OS X applications PyInstaller: Stand-alone executables, under Windows, Linux, Mac OS X, Solaris and AIX Deployment Solid Python Deployments for Everybody Fabric Using Fabric in a development environment Warts and Gotchas Python 2.x gotcha's and landmines Python Warts Python Gotchas Common Pitfalls in Python Web Python and Realtime Web List of Python API's Frameworks Flask Flask Mega Tutorial Series Designing a RESTful API with Python and Flask Building websites in Python with Flask How I structure my Flask apps Designing a RESTful API using Flask-RESTful Rapid Web Prototyping with Lightweight Tools Build a Simple, Static, Markdown-Powered Blog with Flask How to Build an API with Python and Flask Create a Simple Music Streaming App with Flask Web2Py Building a Twitter clone in Web2Py Django The official Django documentation Getting started with Django Building a hacker news clone in Django Djen of Django Deploying Django Django by Example Django From the Ground Up (video tutorials) Tango with Django Django Screencasts Django runserver and stunnel for testing HTTPS Scaling database with Django and HAProxy Classy Class-Based Views Bottle Developing with Bottle- Part 1 Tornado Tornado official documentation Fastest database for Tornado API and Web Services How to access various web services in python Python API for various web services Glassdoor API Scraping Web Scraping with BeautifulSoup Web Scraping 101 with Python Web Scraping with Scrapy Framework How to automatically search and download torrents with Python and Scrapy Mobile Development Kivy Creating an app in Kivy Google Glass Resources The official Python Quick Start for Google Glass Libraries glass.py Create python web applications for Google Glass Image Processing Python Image Library - Examples GUI Programming Python.org GUI Overview Page PyQt Tutorial Standalone executable for PyQt4 on Windows Kivy GUI for cross-platform applications EffBot.org Introduction to Tkinter Pytkgen: JSON to Tkinter GUI Generator Intro to programming with Python and Tkinter Android SL4A Making the Physics Gizmo app for Android using SL4A WSGI Getting started with WSGI WSGI Basics Databases SQLAlchemy SQLAlchemy SQLAlchemy and you Data Mining Linux system mining using Python Data Analysis Pandas A 10 min intro to Pandas Pandas and Python: Top 10 Learn Pandas IPython Notebooks The Traveling Salesperson Problem Scientific Computing with Python Gaussian Blur Understanding the FFT Algorithm lifelines: Survival Analysis in Python Economics Simulation An Introduction to Bayesian Methods for Multilevel Modeling The Floods of the Nile Animal Foraging and the Evolution of Goal-Directed Cognition Wikipedia data scraping functions Python for Geosciences Media Signal Processing Digital Filters Image Filtering Audio Features Feature Detection and Matching for Images Gestures Design Patterns Python Best Practice Patterns by Vladimir Keleshev Factory Pattern In Python Concurrency Patterns Wasp's Nest: The Read-Copy-Update Pattern In Python Concurrency and Distributed Systems Gevent and ZeroMQ Celery: Distributed Task Queue Gevent for the working Python developer Python 2 vs 3 Python 2 or 3 ? Porting to Python 3 How to support both Python 2 and 3 How CherryPy supports Python 2 and 3 ($) Porting to Python 3 - The Book Site Books Free Some free books on Python Invent with Python Think Python Byte of Python Snake Wrangling for Kids Data Structures and Algorithms with Object-Oriented Design Patterns in Python Python programming on wikibooks Dive Into Python 3 Programming Computer Vision with Python Python for you and me Program Arcade Games With Python And Pygame Paid Some list of paid books on python Porting to Python 3 - The Book Site Online Courses and Challenges A Gentle Introduction to Programming Using Python Python Challenges Project Euler Discussions What's your favorite programming language? What is your primary programming language? StackOverflow - Python beyond the basics StackOverflow - Python progression path from apprentice to guru StackOverflow - Zen of Python StackOverflow - Hidden Features of Python Conferences and events Official PyCon website Lanyrd - Python Conference and Events Meetup.com - Python Meetup Groups Videos Guido Van Rossum on the history of Python PyVideo Khan Academy - Introduction to programming and computer science using Python Lanyrd - Python Videos Python Videos, Tutorials and Screencasts at ShowMeDo Talks by Brandon Rhodes A Billion Rows per Second: Metaprogramming Python for Big Data The newboston site tutorial Editors and IDE's for Python programming Setting up Sublime Text for Python Development Sublime Text 2 Love by Kenneth Reitz Sublime Text 2 tips for Python and web developers PyCharm Geany Geany with pylint PyDev Ninja-IDE Emacs setup for python development environment Emacs for python BigData Mind map of Python for Big Data Curated Python resources from other websites Python Resources (All) - Codesters.org Newsletters Pycoder's Python Weekly Miscellaneous Good to great Python reads How to Create Barcodes in Your PDFs with Python What are some good Python interview questions ? Python innards introduction Projects - A list of practical projects More Python Tutorials Simple Sudoku solver in python Looking Inside Dropbox \u2013 Security whitepaper Why sorting an array makes a Python loop faster Python Integer object implementation The history of Python - Blog The ever useful and neat subprocess module Hidden Features in Python Python module of the week Become a programmer Python tutor Best developer blogs about python - Quora Python blogs -Stackoverflow Abstracting Control Flow Python Algorithm Visualizations done in Python running in the browser Solving Every Sudoku Puzzle IHeartPy! >>> source","tags":"pages","loc":"http://yasoob.github.io/blog/pycrumbs","title":"Contents"},{"text":"Python 3 is five years old! November 2013 Annoucing Django Weekend Cardiff, the UK's first Django conference November 2013 Django Rest Framework 2.3.9 Released November 2013 Twisted 13.2.0 Released November 2013 gevent 1.0 released (Concurrency in Python) November 2013 The dates for EuroPython 2014 have been announced November 2013 Celery 3.1 Has Been Released! November 2013 PyPy 2.2.1 - Incrementalism.1 Released November 2013 PyData NYC Nov 8 - 10 November 2013 Announcing the first ever Python Indie Bundle sale! November 2013 Python 3.4.0 beta 1 has been released! November 2013 PEP 428 -- The pathlib module has been approved November 2013 PEP 3156 - Asynchronous IO Support (AKA Tulip) has been accepted November 2013 Python 3.3.3 final released! November 2013 PiCloud has joined Dropbox, PiCloud Platform to continue as open source project November 2013 Bokeh 0.3 released: Interactive web plots for Python November 2013 Python, the most popular programming languages of 2013 November 2013 PyPy 2.2 Incrementalism released November 2013 CherryMusic 0.29.0 released: Stream your music to all your devices! November 2013 PyDev 3.0 released November 2013 Django 1.6 released November 2013 PyTennessee 2014 Schedule Announced November 2013 TurboGears 2.3.1 released today November 2013 Python 2.6.9 final available - The final 2.6X release October 2013 Python3 to become the default Python in Fedora October 2013 Pip will be included in Python 3.4 (PEP 453 -- Explicit bootstrapping of pip in Python installations) October 2013 Python 3.4.0a4 is out! October 2013 R's popular plotting system Ggplot2 ported to python October 2013 NHS drops Oracle for Python, Riak, Redis, RabbitMQ, Tornado, Flask and more October 2013 PyCharm 3.0 Community Edition October 2013 Pymongo 2.6.3 released October 2013 Announcing the Pycon 2014 Keynotes October 2013 Pillow 2.2.1 Released October 2013 PyTennessee Call For Papers! September 2013 PyCon 2014 Talk and Tutorial Call Closes, Reviews Begin September 2013 PyDev 2.8.2 released. Plans for PyDev 3 September 2013 Microsoft donates $100,000 to the IPython team September 2013 Django security releases issued for v1.4.8, v1.5.4 and 1.6 beta 4 September 2013 PyCon 2014 Registration Now Open! September 2013 Requests 2.0.0 released September 2013 Roadmap to Scipy 1.0 released September 2013 Python 3.4.0a2 is out September 2013 Financial Aid Program for PyCon 2014 Is Now Open! August 2013 Pip 1.4.1 Released! August 2013 IPython 1.0 is finally released August 2013 Pylint 1.0 released! August 2013 Gunicorn 18.0 has been released with many bug fixes August 2013 TurboGears 2.3.0 is now on PyPi and faster release cycle announced August 2013","tags":"python, news","loc":"http://yasoob.github.io/blog/python-news","title":"Python News"},{"text":"Hi there guys! I hope all of you are fine and doing well. Recently I was hanging out on a python related IRC where I got a request from someone to write an article on decorators in Python. It is perhaps one of the most difficult concept to grasp. So as usual without wasting anytime let get on with it. - Everything in python is an object (Functions too!): First of all let's understand functions in python: def hi ( name = \"yasoob\" ): return \"hi \" + name print hi () #output: 'hi yasoob' #We can even assign a function to a variable like greet = hi #We are not using parentheses here because we are not calling the function hi #instead we are just putting it into the greet variable. Let's try to run this print greet () #output: 'hi yasoob' #lets see what happens if we delete the old hi function! del hi print hi () #outputs: NameError print greet () #outputs: 'hi yasoob' - Defining functions within functions: So those are the basics when it comes to functions. Lets take your knowledge one step further! In Python we can define functions inside other functions. You might be wondering what sorcery is this! Let me explain it with an example. def hi ( name = \"yasoob\" ): print \"now you are inside the hi() function\" def greet (): return \"now you are in the greet() function\" def welcome (): return \"now you are in the welcome() function\" print greet () print welcome () print \"now you are back in the hi() function\" hi () #output:now you are inside the hi() function # now you are in the greet() function # now you are in the welcome() function # now you are back in the hi() function # This shows that whenever you call hi(), greet() and welcome() # are also called. However the greet() and welcome() functions # are not available outsite the hi() function e.g: greet () #outputs: NameError: name 'greet' is not defined So now we know that we can define functions in other functions. In simpler words we can make nested functions. Now you need to learn one more thing that functions can return functions too. - Returning functions from within functions: It is not necessary to execute a function within another function, we can return it as an output as well. Let's take a look at it with an example! def hi ( name = \"yasoob\" ): def greet (): return \"now you are in the greet() function\" def welcome (): return \"now you are in the welcome() function\" if name == \"yasoob\" : return greet else : return welcome a = hi () print a #outputs: <function greet at 0x7f2143c01500> #This clearly shows that `a` now points to the greet() function in hi() #Now try this print a () #outputs: now you are in the greet() function Just take a look at the code again! In the if/else clause we are returning greet and welcome, not greet() and welcome(). Why is that? It is so because when you put parentheses around it the function gets executed whereas if you don't put parenthesis around it then it can be passed around and can be assigned to other variables without executing it. Did you get it ? Let me explain it a little bit in more detail. When we write a = hi() hi() gets executed and because the name is yasoob by default, the function greet is returned. If we change the statement to a = hi(name = \"ali\") then the welcome function will be returned. We can also do print hi()() which outputs now you are in the greet() function . I hope you have not fainted by now. Was it difficult ? No? I guess you are in the mood to finally learn about decorators! Lets continue our talk and move forward. - Giving a function as an argument to another function: Lets take a look at an example: def hi (): return \"hi yasoob!\" def doSomethingBeforeHi ( func ): print \"I am doing some boring work before executing hi()\" print func () doSomethingBeforeHi ( hi ) #outputs:I am doing some boring work before executing hi() # hi yasoob! Congratulations! You have all of the knowledge to entitle you as a decorator wrangler! Oh wait, I still haven't told you what decorators really are. Here is a short definition decorators let you execute code before and after the function they decorate - Writing your first decorator: You have already written your first decorator! Do you know when? In the last example we actually made a decorator! Lets modify the previous decorator and make a little bit more usable program: def a_new_decorator ( a_func ): def wrapTheFunction (): print \"I am doing some boring work before executing a_func()\" a_func () print \"I am doing some boring work after executing a_func()\" return wrapTheFunction def a_function_requiring_decoration (): print \"I am the function which needs some decoration to remove my foul smell\" a_function_requiring_decoration () #outputs: \"I am the function which needs some decoration to remove my foul smell\" a_function_requiring_decoration = a_new_decorator ( a_function_requiring_decoration ) #now a_function_requiring_decoration is wrapped by wrapTheFunction() a_function_requiring_decoration () #outputs:I am doing some boring work before executing a_function_requiring_decoration() # I am the function which needs some decoration to remove my foul smell # I am doing some boring work after executing a_function_requiring_decoration() Did you get it? We just applied the previously learned principles. This is exactly what the decorators do in python! They wrap a function and modify its behaviour in one way or the another. Now you might be wondering that we did not use the @ anywhere in our code? That is just a short way of making up a decorated function. Here is how we could have run the previous code sample using @ . @a_new_decorator def a_function_requiring_decoration (): print \"I am the function which needs some decoration to remove my foul smell\" a_function_requiring_decoration () #outputs: I am doing some boring work before executing a_function_requiring_decoration() # I am the function which needs some decoration to remove my foul smell # I am doing some boring work after executing a_function_requiring_decoration() #the @a_new_decorator is just a short way of saying: a_function_requiring_decoration = a_new_decorator ( a_function_requiring_decoration ) - Decorators Demystified: I hope you now have a basic understanding of how decorators work in Python. They are not something to be afraid of at all. In addition, we can chain two or more than two decorators! For example: def bread ( func ): def wrapper (): print \"</''''''\\>\" func () print \"<\\______/>\" return wrapper def ingredients ( func ): def wrapper (): print \"#tomatoes#\" func () print \"~salad~\" return wrapper def sandwich ( food = \"--ham--\" ): print food sandwich () #outputs: --ham-- sandwich = bread ( ingredients ( sandwich )) sandwich () #outputs: #</''''''\\> # #tomatoes# # --ham-- # ~salad~ #<\\______/> In the following example e-satis really does a great job demonstrating how decorators are used. We can run the previous example as so: @bread @ingredients def sandwich ( food = \"--ham--\" ): print food sandwich () #outputs: #</''''''\\> # #tomatoes# # --ham-- # ~salad~ #<\\______/> The order in which you use decorators matters and can change the whole behaviour of your decorated function if they are not executed in the intended order. Suppose you write this: @ingredients @bread def sandwich ( food = \"--ham--\" ): print food sandwich () #outputs: ##tomatoes# #</''''''\\> # --ham-- #<\\______/> # ~salad~ Nobody wants a sandwich with tomato on the top and salad on the bottom. This demonstrates why you should not mix up the arrangement of the decorators; otherwise you'll pay the price. In our case we paid the price by ruining our beautiful sandwich. I hope you learned something useful from this post! If there's any demand I will cover some more gotchas and features of decorators in the future. Please share your views in the comments below. If you have any suggestions, then please do tell me about them. You can also follow me on facebook, twitter, or you can send me an email. The links to my social media accounts are at the bottom of the post. Farewell, and don't forget to subscribe to my blog, tweet, and share this post on facebook.","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/python-decorators-demystified","title":"Python decorators finally demystified"},{"text":"Recently I've started learning about Kivy, a Python Natural User Interface (NUI) toolkit. As I understand it, Kivy is kind of a spiritual successor to pyMT , which you can read more about here . In this article, we will be learning how Kivy handles layout management. While you can position widgets using x/y coordinates, in every GUI toolkit I've used, it's almost always better to use some kind of layout management that the toolkit provides. This allows the widgets to resize and move appropriately as the user changes the window's size. In Kivy, these things Layouts . If you've used wxPython, they are analogous to wxPython's sizers. I should also note that Kivy can do layouts in two different ways. The first way is to do Layouts with Python code only. The second way is to use a mixture of Python and Kv language. This is to promote the model-view-controller way of doing things. It looks kind of like CSS and reminds me a little of wxPython and XRC. We will look at how to use both methods in this article. While Kivy supports multiple types of Layouts, this article will be focusing only on the BoxLayout . We will show how to nest BoxLayouts. Kivy, Python and BoxLayout Creating a BoxLayout in Kivy using Python is actually pretty easy and quite intuitive. We'll start out with a code example and then follow the code with an explanation. Let's get started! import kivy import random from kivy.app import App from kivy.uix.button import Button from kivy.uix.boxlayout import BoxLayout red = [ 1 , 0 , 0 , 1 ] green = [ 0 , 1 , 0 , 1 ] blue = [ 0 , 0 , 1 , 1 ] purple = [ 1 , 0 , 1 , 1 ] ######################################################################## class HBoxLayoutExample ( App ): \"\"\" Horizontally oriented BoxLayout example class \"\"\" #---------------------------------------------------------------------- def build ( self ): \"\"\" Horizontal BoxLayout example \"\"\" layout = BoxLayout ( padding = 10 ) colors = [ red , green , blue , purple ] for i in range ( 5 ): btn = Button ( text = \"Button # %s \" % ( i + 1 ), background_color = random . choice ( colors ) ) layout . add_widget ( btn ) return layout ######################################################################## class VBoxLayoutExample ( App ): \"\"\" Vertical oriented BoxLayout example class \"\"\" #---------------------------------------------------------------------- def setOrientation ( self , orient ): \"\"\"\"\"\" self . orient = orient #---------------------------------------------------------------------- def build ( self ): \"\"\"\"\"\" layout = BoxLayout ( padding = 10 , orientation = self . orient ) for i in range ( 5 ): btn = Button ( text = \"Button # %s \" % ( i + 1 ) ) layout . add_widget ( btn ) return layout #---------------------------------------------------------------------- if __name__ == \"__main__\" : app = HBoxLayoutExample () #app = VBoxLayoutExample() #app.setOrientation(orient=\"vertical\") app . run () Here we have created a vertically oriented BoxLayout class and a horizontally oriented BoxLayout class. Each class contains 5 buttons with random background colors. The colors follow RGBA, but can have individual values that are between zero and one. Oddly enough, if you use numbers greater than one, the color becomes brighter. I happened to use 255 instead of 1 when I created the screenshot above, so if you happen to run this code and see a more muted set of colors, that's why. To make the examples extremely simple, we only import Kivy's App, Button and BoxLayout classes. The BoxLayout class accepts several arguments, but we'll focus on the following 3: orientation, padding and spacing. Because BoxLayout is a sub-class of Layout and Widget, it inherits many other methods and keyword arguments that are not covered here. But back to the arguments we currently care about. The padding argument tells Kivy how much space there should be between the Layout and its children, whereas the spacing arguments tells it how much spacing there should be between the children. To create the buttons, we use a simple loop that loops over a small range of numbers. Each iteration creates a button with a random background color and adds that button to the Layout instance. Then we return the layout at the end. The vertical BoxLayout example in the VBoxLayoutExample class is slightly different in that I thought it would be fun to be able to set the orientation programmatically. The code is pretty much the same except that I added a setOrientation method. Note that if you call setOrientation again, it will have no effect. As one of my commenters so kindly pointed out, you would need to bind the orientation to the App orient property or use the Kv language to achieve this. If you comment out the call to HBoxLayoutExample at the end of the script and un-comment out the other two lines, then you should end up seeing something like this: Notice that when you don't set a background color, Kivy defaults to a dark grey. Kivy does not try to look like a native application. This may or may not be a big deal to you depending on what sort of program you're trying to achieve, but it should be noted. Now we're ready to learn about nesting! Nesting BoxLayouts Nesting BoxLayouts inside of each other is pretty easy with Kivy too. Whenever you go to create an application with a complicated interface that will need nested sizers, you should take some time to sketch the layout out with pencil and paper. Then you can draw boxes around the widgets in different ways to help you visualize which Layouts you'll need and how to nest them in each other. I have found this quite helpful with wxPython and I think it applies to any other GUI toolkit that doesn't have a WYSIWYG editor. By the way, BoxLayouts are very powerful. If you know what you're doing, you can make just about any interface with them alone just be using clever nesting. Enough talk, let's look at some code! import kivy import random from kivy.app import App from kivy.uix.button import Button from kivy.uix.boxlayout import BoxLayout red = [ 1 , 0 , 0 , 1 ] green = [ 0 , 1 , 0 , 1 ] blue = [ 0 , 0 , 1 , 1 ] purple = [ 1 , 0 , 1 , 1 ] ######################################################################## class NestedLayoutExample ( App ): \"\"\" An example of nesting three horizontally oriented BoxLayouts inside of one vertically oriented BoxLayout \"\"\" #---------------------------------------------------------------------- def build ( self ): \"\"\" Horizontal BoxLayout example \"\"\" main_layout = BoxLayout ( padding = 10 , orientation = \"vertical\" ) colors = [ red , green , blue , purple ] for i in range ( 3 ): h_layout = BoxLayout ( padding = 10 ) for i in range ( 5 ): btn = Button ( text = \"Button # %s \" % ( i + 1 ), background_color = random . choice ( colors ) ) h_layout . add_widget ( btn ) main_layout . add_widget ( h_layout ) return main_layout #---------------------------------------------------------------------- if __name__ == \"__main__\" : app = NestedLayoutExample () app . run () This example is very similar to the last one. The devil is in the details though. Here we have a nested for loop that creates 3 BoxLayouts that contain 5 buttons a piece. Each Layout is then inserted into the top level Layout at the end of each iteration in the outside loop. In case you missed it, scroll back up to see how it turned out. The trick is to create one top-level or main Layout and add other Layouts to it. Now let's turn our attention to learning how to do these things with the Kv language. Kv+Python and BoxLayout It's almost always a little painful to learn a new language. Fortunately, the Kv language actually follows Python pretty closely, including Python's requirement of using indentation levels to denote when a section of code begins and ends. You may want to spend a few minutes reading about the Kv language on the Kivy website . Whenever you're ready, we can continue. First we'll start off with the Python code: # kvboxlayout.py from kivy.app import App from kivy.uix.boxlayout import BoxLayout ######################################################################## class KVMyHBoxLayout ( BoxLayout ): pass ######################################################################## class KVBoxLayoutApp ( App ): \"\"\"\"\"\" #---------------------------------------------------------------------- def build ( self ): \"\"\"\"\"\" return KVMyHBoxLayout () #---------------------------------------------------------------------- if __name__ == \"__main__\" : app = KVBoxLayoutApp () app . run () This code is much simpler than our previous examples, but it's also rather mysterious. First of all, we create an empty sub-class of BoxLayout. Then we create our App class which has a build method that just returns an instance of the empty BoxLayout class. What's going on here? Well we have to look at the Kv file to find out! < MyButton @Button > : color : . 8 , . 9 , 0 , 1 font_size : 32 < MyButton @Button > : orientation : 'horizontal' MyButton : text : \"Btn1\" background_color : 1 , 0 , 0 , 1 MyButton : text : \"Btn2\" background_color : 0 , 1 , 0 , 1 MyButton : text : \"Btn3\" background_color : 0 , 0 , 1 , 1 MyButton : text : \"Btn4\" background_color : 1 , 0 , 1 , 1 MyButton : text : \"Btn5\" background_color : 1 , 0 , 0 , 1 When you save the code above, you'll have to name it to be the same as the App class, but with a .kv instead of a .py and in lowercase. That means the name of this Kv file needs to be kvboxlayout.kv . You will note that you also need to strip off the App part of the class name such that KVBoxLayoutApp becomes kvboxlayout. Yes, it's a little confusing. If you don't follow the naming conventions correctly, the file will run but you will have an empty black window. Anyway, first off in the Kv file, we have a section that starts with : . This tells Kivy that we are sub-classing the Button class and calling our sub-class MyButton . Then we indent the required four spaces and set the button's label color and font size. Next we create a BoxLayout section. Notice that we didn't create a sub-class this time. Then we tell it what orientation it should be and add 5 MyButton instances, each one having its own individual label and color. One of the core Kivy developers pointed out that by creating the BoxLayout in this manner, I am redefining the BoxLayout for all usages. This is not a good thing, even if it does make the example simpler. Thus in the next example, we'll stop doing that and do it the right way instead! Nesting BoxLayouts with Kv Nesting BoxLayouts in Kv is a little confusing at first, but once you get the hang of it, you'll find that it's really quite easy. We'll start out with the Python code, take a look at how it works and then look at the Kv code. from kivy.app import App from kivy.uix.boxlayout import BoxLayout from kivy.uix.widget import Widget ######################################################################## class HBoxWidget ( Widget ): pass ######################################################################## class VBoxWidget ( Widget ): pass ######################################################################## class KVNestedBoxLayoutApp ( App ): \"\"\"\"\"\" #---------------------------------------------------------------------- def build ( self ): \"\"\"\"\"\" return VBoxWidget () #---------------------------------------------------------------------- if __name__ == \"__main__\" : app = KVNestedBoxLayoutApp () app . run () This time around, we need to create two generic Widget classes: HBoxWidget and VBoxWidget. These are actually dummy classes that become BoxLayouts in the Kv code. Speaking of which, let's take a look at that now. Note that you'll need to name the Kv file kvnestedboxlayout.kv , which you'll note, is a lowercase version of KVNestedBoxLayoutApp . : color : . 8 , . 9 , 0 , 1 font_size : 32 : BoxLayout : size : root . size pos : root . pos orientation : 'horizontal' MyButton : text : \"Btn1\" background_color : 1 , 1 , 1 , 1 MyButton : text : \"Btn2\" background_color : 0 , 1 , 0 , 1 MyButton : text : \"Btn3\" background_color : 0 , 0 , 1 , 1 MyButton : text : \"Btn4\" background_color : 1 , 0 , 1 , 1 MyButton : text : \"Btn2\" background_color : 1 , 0 , 0 , 1 : BoxLayout : size : root . size pos : root . pos id : foo_bar orientation : 'vertical' HBoxWidget : HBoxWidget : The button code is the same as before. Next we have the HBoxWidget which we define as a horizontal BoxLayout with 5 buttons in it. Then we create an instance of VBoxWidget that is a vertical BoxLayout, but this Layout contains two instances of the HBoxWidget. You'll note that in the Python code's build method, we are returning the VBoxWidget, so that's where the action is. If you remove those two HBoxWidget calls, the result will be an empty black window. There is another way to use Kv files in Kivy. It is through the kivy.lang.Builder.load_file (or load_string) API, which gives you the ability to load Kv files without needing to remember to name the Kv file in some special way. You can read about the API on their website and see an example of it in action in the Kivy examples on github . The only caveat to using this method is that you need to be careful not to load the same file twice or your UI may get messed up. Wrapping Up This just scratches the surface of Kivy's Layout system. There are 6 other Layout types available. However, I think you'll find that the examples in this article will get you started down the road of successfully creating cool Kivy applications of your very own. If you need help learning Kivy, there's a pretty good set of documentation on their website. They also have a Google Group and a #kivy channel on freenode. Related Readings Kivy's Getting Started with Layouts Kivy's programming guide also has coverage on layouts A simple nested Layout example on github Kivy's Kv language page Other Kivy examples on github Download the Source kivy_box_layouts.tar kivy_box_layouts.zip Note: This is an official cross-posting of an article from the Mouse Vs Python blog . You can read the original here .","tags":"news, python","loc":"http://yasoob.github.io/blog/kivy-101-how-to-use-boxlayouts","title":"Kivy 101: How to Use BoxLayouts"},{"text":"Hi guys! In todays post we will be looking into myspace.com and check whether it is possible to download songs with python or not. If it is possible then we will make a python script which will assist us in downloading the songs. I guess you are as eager to start as I am. So lets get started. However let me make it clear in the beginning that this is only for educational purposes. Tools : python - I am using py2k browser - I am using chrome rtmpdump & rtmpsrv internet connection Lets start : So lets start. Lets go to myspace: We don't need this page. Lets go to a song page for instance I am going to open this page . Now lets check out the source of the page to see whether we can find any mp3 link. What I usually do is that I right click on the name of the song and click on \"Inspect this\". Usually the name is in the song link on most websites. So just do this and see if you get anything. Here is what I got: Can you see that data stream url ? Oh yes the one which I have highlighted. That is an rtmp url. RTMP is a streaming protocol. Now the problem is that we can not directly download the rtmp links so what should we do now ? We are very lucky because there is already a software which can help us in downloading this and it is called rtmpdump However rtmpdump requires some additional parameters for downloading the song. They are the play path, stream url, file name and etc. So how do we get those ? So again we are very lucky because there is a software for this as well and it is developed by the same people who have developed rtmpdump. This one is called rtmpsrv Just search for it on google and download it. This is like a proxy. It scans all of your data and searches for rtmp streams. When it has found a stream it just outputs a single command which can be used to download a file with rtmpdump. However first we have to route all the tcp traffic to port 1935. Thats the port where rtmpsrv runs. In order to do so just type this in the terminal: sudo iptables - t nat - A OUTPUT - p tcp -- dport 1935 - j REDIRECT This will add a firewall rule. Now go to myspace and play a song. After you hit play and wait for some seconds then you will see that a couple of lines are outputted by rtmpsrv. Check this out: Now if you copy this command and paste it in the terminal you will see that rtmpdump will start to download the file. Now close rtmpsrv and revert the firewall rule by using this command : sudo iptables - t nat - D OUTPUT - p tcp -- dport 1935 - j REDIRECT Now comes the main part. It is not convenient to go through all of these steps again and again just to download a single song. Now we need to automate this and for that we will use python. After doing a little bit of tinkering I saw that only the -y and -p parameters are changed for every song. Further down the lane we can see that the -p parameter is just the url of the song page and the -y parameter is taken from the rtmp link which we found on the song page. Writing Python script : So lets first make a script which will get the rtmp link from the page. Here is my solution: import requests as r import re html = r . get ( song_page_url ) rtmp_link = re . findall ( 'data-stream-url=\"(.+?)\"' , html . text )[ 0 ] print rtmp_link So now we have a script which gets the rtmp link. Now we need to modify this script so that it outputs the required rtmpdump command. Again here is my solution : import requests as r import re html = r . get ( song_page_url ) rtmp_link = re . findall ( 'data-stream-url=\"(.+?)\"' , html . text )[ 0 ] y = rtmp_link . split ( ';' )[ 1 ] name = ' ' . join ( song_page_url . split ( '/' )[ - 1 ] . split ( '-' )[: - 2 ]) rtmp_command = 'rtmpdump -r \"rtmpte://fms.ec-music.myspacecdn.com/\" \\ -a \"\" -f \"LNX 11,9,900,152\" -o \"{}.flv\" \\ -W \"http://lads.myspacecdn.com/music/sdkwrapper/SDKWrapper.2.2.16.swf\" \\ -p \"http://www.myspace.com\" -y \"{}\"' . format ( name , y ) So now we just need to open rtmpdump with the parameters. For that I will be using the subprocess module. Here is the final code: import re import sys import requests import subprocess song_page_url = sys . argv [ 1 ] html = requests . get ( song_page_url ) . text match = re . search ( 'data-stream-url=\"(rtmpe://[^;]+);([^\"]+)\"' , html ) rtmpurl , playpath = match . group ( 1 , 2 ) filename = ' ' . join ( song_page_url . split ( '/' )[ - 1 ] . split ( '-' )[: - 2 ]) + '.flv' print 'downloading' , filename subprocess . call ([ 'rtmpdump' , '-o' , filename , '-r' , rtmpurl , '-y' , playpath ], stderr = subprocess . DEVNULL ) Just run this file like this : $ python myspace . py https: //m yspace . com /brunomars/m usic /song/som ewhere - in - brooklyn - 69059954 - 74825216 Thats it! We now have a simple script that will download myspace songs for us. I know you are thinking about more features in this script. Let me give you some ideas. This script downloads just songs and does not have any input validation in place. Maybe you can make a similar script for videos ? When you make something do let me know. That was todays tutorial. I will see you later and don't forget to follow this blog, like it on facebook , follow me on twitter and reply below :) This is the best way to repay me. Lastly stay tuned for the next post. Disclaimer: Some of the above code can be written more compactly. I wrote it this way so that the beginners could understand what is going on.","tags":"installation, python, tutorial","loc":"http://yasoob.github.io/blog/reverse-engineering-myspace","title":"Reverse engineering myspace"},{"text":"Hi there folks. I recently thought that I have not written even a single post about testing in python. Testing is one of the most important part of any language. In this post I am going to share some information about unittest with you. So what exactly is unittest ? You might have heard about it on SO or some other forum. It is a testing framework for python just like Junit for Java. It comes pre-installed with python from 2.1. It is easy to use. However there are a lot of other testing frameworks for python out there as well but I will be focusing on unittest today as it is the default testing framework for python. So without wasting any time lets get started. The standard workflow while using unittest is: derive your own class from unittest.TestCase write your tests in functions which start with \"test_\" finally write unittest.main() at the end of your file to run the tests I was also used to be scared by testing. Testing seemed to me as something from outer space but now I have learned its importance and it is essential for every programmer to learn it. A simple script So lets write a simple script which we can later test. This script is going to do some math functions for us. So here is the script: # save it as math.py def multiply ( n1 , n2 ): return n1 * n2 def add ( n1 , n2 ): return n1 + n2 So thats our little script. Now lets move forward and write our first test. Our first test So as I told you before that every unittest file contains a custom class derived from unittest.TestCase so lets create that: import unittest from math import multiply , add class TestingMath ( unittest . TestCase ): pass if __name__ == \"__main__\" : unittest . main () So that was the first part. Now we need to define our tests. In unittest there is a setUp() and tearDown() function. The setUp() function is used to set up the test environment and tearDown() is used to clean up after a test. We do not need them as they are usually used when a lot of tests are written for a larger software. The default implementation of setUp() and tearDown() does nothing. So now lets write our first test by editing our previous script. import unittest from math import multiply , add class TestingMath ( unittest . TestCase ): def setUp ( self ): pass def test_multiply ( self ): self . assertEqual ( multiply ( 3 , 5 ), 15 ) def test_add ( self ): self . assertEqual ( add ( 3 , 5 ), 8 ) if __name__ == \"__main__\" : unittest . main () Now save this file as test.py and run it from the command line and you will see some output like this: yasoob @yasoob: ~/ Desktop $ python test . py .. ---------------------------------------------------------------------- Ran 2 tests in 0.000 s OK Congratulations! You have written your very first fully working test suite in python. Wait! what are those assertEqual() statements there ? Let me explain them. These assert funtions are the backbones of testing in unittest. They check whether the result is correct or not. There are a lot of assert functions in unittest. In our case we used assertEqual() which checks whether two values are equal or not. We gave 2 parameters to assertEqual() now the job of assertEqual() is to check whether both parameters are equal or not. Just for reference some other assert functions are: assertTrue(x) assertFalse(x) assertNotEqual(a, b) assertIs(a, b) assertIsNot(a, b) assertIsNone(x) Test Discovery So just think about it for a minute. You have a lot of test files and want to run all of them. The only method which comes to mind is to manually run all of those files separately. This is possible if you have 10 files but what if you have a 100 files ? That is were automatic test discovery comes to rescue. So if I have all of my tests in app/tests directory I would simply run: python - m unittest discover app / tests So this command would gather all of the test files in the tests directory and will run them. I hope that was helpful. Furthur So now you have got some idea of how testing works in python. Remember that this was just an introduction and there is a whole lot to unittest than this. If you want to further increase your knowledge then check out the original python docs about unittest . I will cover other testing frameworks as well in the future. I hope you enjoyed this article and do remember this one tip that the best time to write tests is while developing. If you write tests bit by bit while developing then they will not become a burden and your application will be rock solid. Do share your views in the comments below or feel free to email me or tweet me . Last but not the least follow this blog and stay tuned for the next post.","tags":"tutorial","loc":"http://yasoob.github.io/blog/introduction-to-unittest","title":"Introduction to unittest"},{"text":"Hi there friends. If you are a python programmer then there is not a single chance that you don't know Guido. Everyone knows him in the python community. However for those who don't know about him let me introduce him to you. He is the creator of python and is known as BDFL. Now a days he works at dropbox and previously he used to work at google. In this post I am going to share some of his recent posts with you. They are very informative and I suggest that you take a look at them. The History of Python: The story of None, True and False (and an explanation of literals, keywords and builtins) In this post Guido tells us about None, True and False and how they became what they are right now in python. Why Python uses 0-based indexing Ah so if you are a new programmer then you might have thought why python as a lot of other languages uses 0-based indexing. There is a reason for everything and so in this post Guido tells us the reason for why python uses 0-based indexing. Python 2 vs. Python 3: A Restrospective The name is self explanatory. This presentation is really useful and you should definately go through it. Guidos' SF presentation: Tulip: Async I/O for Python 3 @ SF Python Meetup In this presentation guido tells about Tulip, Asynchronous I/O for python 3. If you ever wanted to contribute to a core python library but did not know what to contribute then do take a look at this as this might be a good chance for you. So now let me say goodbye to you. I hope you enjoyed this article as much as I enjoyed writing it. I will see you guys later. Don't forget to follow me on twitter and follow this blog as well. And finally stay tuned for the next post as it will be great.","tags":"inspiration, news","loc":"http://yasoob.github.io/blog/recent-posts-by-guido-van-rossum","title":"Recent posts by Guido Van Rossum"},{"text":"Hi folks welcome back to yet another article. This one is going to be short. In this post I will be sharing two new books which have recently come out. These books are regarding GUI development in Python. In past several months we have not seen a lot of books focused towards GUI development in Python. Hopefully these two books will fill the gap. So without wasting any time lets look at them. * Tkinter GUI Application Development Hotshot : Its been a long time since we saw a new Tkinter book out there. This book has received some good critics by the professional python programmers. According to the overview given on the website you will : Develop 10 real GUI applications using procedural and object-oriented styles Write network programs, database-driven programs, multi-threaded programs and more with Tkinter A practical guide to discover the best practices involved in writing GUI applications * Test-Driven Development with Python : Test-Driven Development with Python focuses on web development, with some coverage of JavaScript. This book uses a concrete example\u2014the development of a website, from scratch\u2014to teach the TDD metholology. This book is not completed as yet however most of it has been completed. You can read this book online for free or buy it from O'Reilly. So now we come to an end of yet another useful post. Do follow me on twitter , follow this blog and comment below to share your views and to give me an idea about a future post. I will be waiting for your comments.","tags":"news","loc":"http://yasoob.github.io/blog/new-python-books","title":"New Python books for GUI development"},{"text":"Hi there guys. So a lot has happened while I was away. It has been only a few weeks and a whole ton of great python project have emerged. In this post I will share a list of some of the excellent projects I saw in the past few days. So lets take a look at them without wasting any more time. Lans.py This project is interesting mostly for those who are into network security. This is a single script providing a set of useful commands. It is a packet parsing/injecting arp spoofer. Check out the github page for more information CherryMusic CherryMusic is not a new project but it recently got a new release. It is basically a music streaming server written in python. It allows you to stream your own music collection to all your devices! I have personally tested it and really recommend it. Python Wheels Wheels are the new standard of python distribution and are intended to replace eggs. Among other advantages, they offer faster installation and allow secure digital signing. This project does not support a lot of python packages at the moment but they are trying to support as many projects as possible. Dataset dataset provides two key functions that make using SQL databases in Python a breeze. A simple abstraction layer removes most direct SQL statements without the necessity for a full ORM model and Database contents can be exported ( frozen ) using a sophisticated plain file generator with JSON and CSV support. Openpyxl OpenPyxl is a Python library to read/write Excel 2007 xlsx/xlsm files. It was born from lack of existing library to read/write natively from Python the new Open Office XML format. Do check this project if you deal a lot with Excel files. You never know when you find something useful If you are still reading then I guess you are hungry for more projects. Never mind I will do more posts about such things. For the time being do follow me on twitter , follow this blog and comment below to share your views and to give me an idea about a future post.","tags":"help, inspiration, news, python","loc":"http://yasoob.github.io/blog/some-python-projects-worth-looking-at","title":"Some Python projects worth looking at"},{"text":"Hi there folks. Its been a long time since I posted here. That was because my studies were giving me some problems. So no worrys because the problems are all resolved and I am back with yet another news. So what is PyDev ? If you have been coding python scripts with eclipse (it's an IDE) then the chances are that you know what PyDev is . So for those who are not familiar with pydev read on. PyDev is a third-party plug-in for Eclipse. It is an Integrated Development Environment used for programming in Python supporting code refactoring, graphical debugging, code analysis and many other features. Recently it's 3.0 version was released after a long break. Some believed that the development for PyDev had stopped but this release has given hope to eclipse lovers. According to the official pydev website PyDev comes with many goodies such as: Django integration Code completion Code completion with auto import Type hinting Code analysis Go to definition Refactoring Debugger Remote debugger Tokens browser Interactive console Unittest integration Code coverage and many others : I believe it is worth to check out this project. The link to the official website is http://pydev.org/ and here is the link to this release announcement . Do share your views in the comments below and stay tuned for the next post. If you know someone who can benefit from my post then do make sure that you tell them about this blog.","tags":"news, python","loc":"http://yasoob.github.io/blog/pydev-3-0-released","title":"PyDev 3.0 Released"},{"text":"Hi there Pythonistas. A news has arrived that pip will be available by default with python 3.4. PEP 453 which was \"Explicit bootstrapping of pip in Python installations\" has been accepted. Python 3.4 which has entered into the beta phase after the release of final alpha will have pip by default. If you are new to python then you might be wondering what pip is ? Pip is a tool for installing and managing Python packages. Previously easy_install has been the most used package manager. Easy_Install is a python module ( easy_install ) bundled with setuptools that lets you automatically download, build, install, and manage Python packages but has a lot of weak points in front of pip. In order to use pip you first have to install it by using easy_install . Why pip is better: The command is simpler, shorter First-class support of virtualenv More commands than just install , including uninstall . pip bundle , pip freeze and pip search are pretty nice as well. Can install from a VCS (via -e ), or from source Requirements files are easy to use, clearly describes what happens in its process If it can not download all the dependencies, it will not install anything (though I don't think it rolls stuff back if an installation fails I hope you are as much happy as I am with the acceptance of this PEP. Do share your views in the comments below and don't forget to share this post and follow this blog in order to get the latest news relating with the python world.","tags":"news, python","loc":"http://yasoob.github.io/blog/python-3-4-to-be-equipped-with-pip-by-default","title":"Python 3.4 to be equipped with pip by default"},{"text":"The makers of the hugely popular Python IDE PyCharm recently announced a community edition of their Python IDE. JetBrains announced that they will be offering a community Edition of PyCharm however it will have some stripped down features for example it will not have builtin support for pyramid etc. However the news for a Community version of PyCharm became hot in the python community recently because PyCharm is currently the best IDE available for Python. This community edition has been out for quite some time now and has already been downloaded by a lot of developers. Today the Source code of this IDE was also open sourced by JetBrains. The source code is available on github . This IDE has a lot of cool features. Here are a few of the features of the community version: Intelligent Editor, with code completion, on-the-fly error highlighting, auto-fixes, etc. Automated code refactorings and rich navigation capabilities Integrated debugger and unit testing support Native VCS integrations Customizable UI and key-bindings, with VIM emulation available And much more, all available under the Apache 2 license I hope you are as much excited as I am. I have already taken this IDE for a test drive and have been fascinated by it's power. It is a one stop solution for every python developer whether he is a system engineer or a web developer. I hope you will take this IDE for a test drive. Do share your experience in the comments below so that others can know about your experience and don't forget to follow this blog if you want to get latest news, tips, tricks and tutorials relating to python.","tags":"news, python","loc":"http://yasoob.github.io/blog/pycharm-3-0-community-edition","title":"PyCharm 3.0 Community Edition"},{"text":"Hi folks in this post i am going to share with you news from the python world. This is like a roundup of monthly news. Latest News 1. NHS drops Oracle for Python, Riak, Redis, RabbitMQ, Tornado, Flask and more The UK government's quest to get public services to use more open source technologies seems to be taking hold, judging by the revamp of the NHS's very large Spine service. 2. Microsoft donates $100,000 to the IPython team Microsoft donates $100,000 to the IPython team for the continued development of IPython. 3. PyCharm 3.0 Community Edition Jet Brains recently surprised the Python community by launching a community version of it's extremely popular Python IDE. It's a must have for a Python programmer. 4. PyMongo 2.6.3 Released MongoDB is one of the leading NoSQL Database. The PyMongo distribution contains tools for interacting with MongoDB database from Python. Python Conferences: 1. PyCon 2014 Keynotes announced The keynote speakers for the largest Python Conference have been announced. This year the keynote speakers include Van Lindberg, Jessica McKellar, Fernando Perez, Guido Van Rossum and John Perry Barlow. 2. PyTennessee Call for Papers! If you are in the Tennessee area and know python then make sure that you do not miss this great chance to attend the PyTennnesse. Submit your papers now. The deadline is 1st November. Articles: 1. Breaking Out of Secured Python Shells Ever thought of hacking secure python shells ? I hope not but this guy definitely thought about that and even accomplished. Hop over to his website to read about his discovery. It is definitely worth a read. 2. Cubr! A really cool project. This one involves the use of a mixed up Rubik's Cube. Just bring the mixed layout of a Rubik's Cube in front of your webcam and let this nifty application do it's trick. 3. Mapping data in Python using Pandas and Vincent This Blogpost clearly shows the power of Vincent. Do check it out if you are into data mining and data representation. This Blogpost will surely help you. 4. Training Random Forests in Python using the GPU Random Forests have emerged as a very popular learning algorithm for tackling complex prediction problems. These guys have released GPU Random Forest library for Python called CudaTree. Check this out if you want to learn more. 5. Introduction to Hardware Hacking with Arduino There are not a lot of post out there which explain clearly how to hack an Arduino using Python and turning it into a useful creature. This post is for Hardware Hackers who are a bit familiar with Python. 6. Why are there so many Pythons ? This post starts from scratch and takes you step by step into the world of different Python implementations. Read it if you have time. 7. Programming a Perceptron in Python A perceptron classifier is a simple model of a neuron. In this post the author programs a Perceptron in Python. 8. Read and Write Video Frames in Python Using FFMPEG This article shows how easy it is to read or write video frames with Python, by calling the external software FFMPEG through pipes. Worth the read if you are interested in video manipulation. Interesting Projects: 1. csscompressor -- A python port of YUI CSS Compressor Port of YUI CSS Compressor to Python 2. CIntruder Captcha Intruder is an automatic pentesting tool to bypass captchas. 3. Arrow 0.4 Better dates & times for Python, now with ISO-8601 parsing and week support! 4. Lifelines Survival analysis in Python. 5. Pymultitor With this algorithm you can use multiple tor threads to make multiple requests with multiple IP addresses. 6. Flask-Sockets: Easy WebSockets in Flask, by Kenneth Reitz This guy is super nice and always contribute useful libraries. This one is no less. Check this out if you are working with websockets. 7. Whoosh - a fast, pure Python search engine library. Really a wonderful project. Whoosh is a fast, pure Python search engine library. Whoosh is not really a search engine, it's a programmer library for creating a search engine 8. Centrifuge - real-time messaging in web applications Simple real-time messaging in web applications.The main goal of Centrifuge is the same as of Pusher or Pubnub services. The main difference is that Centrifuge is open-source and requires installation. I hope you liked this roundup. Make sure that you share this post on Facebook and Twitter. Stay tuned for the next post! Also do follow my blog so that you do not miss a single post and finally comment below if I missed something and if you think that I should do similar posts in the future as well.","tags":"news, python","loc":"http://yasoob.github.io/blog/python-news-13-oct-2013","title":"Python News 13 Oct 2013"},{"text":"Hi there fellas. This is an answer posted on stackoverflow by e-satis. The original link to the answer is given at the end. No credit goes to me. All of the credit goes to the original author. This answer is posted just because most of us are unaware of how decorators work in python and this answer solves that problem beautifully. Python's functions are objects To understand decorators, you must first understand that functions are objects in Python. This has important consequences. Let's see why with a simple example : def shout ( word = \"yes\" ): return word . capitalize () + \"!\" print shout () # outputs : 'Yes!' # As an object, you can assign the function to a variable like any # other object scream = shout # Notice we don't use parentheses: we are not calling the function, we are # putting the function \"shout\" into the variable \"scream\". # It means you can then call \"shout\" from \"scream\": print scream () # outputs : 'Yes!' # More than that, it means you can remove the old name 'shout', and # the function will still be accessible from 'scream' del shout try: print shout () except NameError , e: print e #outputs: \"name 'shout' is not defined\" print scream () # outputs: 'Yes!' OK, keep that in mind, we are going back to it soon. Another interesting property of Python functions is they can be defined... inside another function! def talk (): # You can define a function on the fly in \"talk\" ... def whisper ( word = \"yes\" ): return word . lower () + \"...\" # ... and use it right away! print whisper () # You call \"talk\", that defines \"whisper\" EVERY TIME you call it, then # \"whisper\" is called in \"talk\". talk () # outputs: # \"yes...\" # But \"whisper\" DOES NOT EXIST outside \"talk\": try: print whisper () except NameError , e: print e #outputs : \"name 'whisper' is not defined\"* Functions references OK, still here? Now the fun part, you've seen that functions are objects and therefore: can be assigned to a variable; can be defined in another function. Well, that means that a function can return another function :-) Have a look: def getTalk ( type = \"shout\" ): # We define functions on the fly def shout ( word = \"yes\" ): return word . capitalize () + \"!\" def whisper ( word = \"yes\" ) : return word . lower () + \"...\" ; # Then we return one of them if type == \"shout\" : # We don't use \"()\", we are not calling the function, # we are returning the function object return shout else : return whisper # How do you use this strange beast? # Get the function and assign it to a variable talk = getTalk () # You can see that \"talk\" is here a function object: print talk #outputs : <function shout at 0xb7ea817c> # The object is the one returned by the function: print talk () #outputs : Yes! # And you can even use it directly if you feel wild: print getTalk ( \"whisper\" )() #outputs : yes... But wait, there is more. If you can return a function, then you can pass one as a parameter: def doSomethingBefore ( func ): print \"I do something before then I call the function you gave me\" print func () doSomethingBefore ( scream ) #outputs: #I do something before then I call the function you gave me #Yes! Well, you just have everything needed to understand decorators. You see, decorators are wrappers which means that they let you execute code before and after the function they decorate without the need to modify the function itself. Handcrafted decorators How you would do it manually: # A decorator is a function that expects ANOTHER function as parameter def my_shiny_new_decorator ( a_function_to_decorate ): # Inside, the decorator defines a function on the fly: the wrapper. # This function is going to be wrapped around the original function # so it can execute code before and after it. def the_wrapper_around_the_original_function (): # Put here the code you want to be executed BEFORE the original # function is called print \"Before the function runs\" # Call the function here (using parentheses) a_function_to_decorate () # Put here the code you want to be executed AFTER the original # function is called print \"After the function runs\" # At this point, \"a_function_to_decorate\" HAS NEVER BEEN EXECUTED. # We return the wrapper function we have just created. # The wrapper contains the function and the code to execute before # and after. It's ready to use! return the_wrapper_around_the_original_function # Now imagine you create a function you don't want to ever touch again. def a_stand_alone_function (): print \"I am a stand alone function, don't you dare modify me\" a_stand_alone_function () #outputs: I am a stand alone function, don't you dare modify me # Well, you can decorate it to extend its behavior. # Just pass it to the decorator, it will wrap it dynamically in # any code you want and return you a new function ready to be used: a_stand_alone_function_decorated = my_shiny_new_decorator ( a_stand_alone_function ) a_stand_alone_function_decorated () #outputs: #Before the function runs #I am a stand alone function, don't you dare modify me #After the function runs Now, you probably want that every time you call a_stand_alone_function , a_stand_alone_function_decorated is called instead. That's easy, just overwrite a_stand_alone_function with the function returned by my_shiny_new_decorator : a_stand_alone_function = my_shiny_new_decorator ( a_stand_alone_function ) a_stand_alone_function () #outputs: #Before the function runs #I am a stand alone function, don't you dare modify me #After the function runs # And guess what? That's EXACTLY what decorators do! Decorators demystified The previous example, using the decorator syntax: @my_shiny_new_decorator def another_stand_alone_function (): print \"Leave me alone\" another_stand_alone_function () #outputs: #Before the function runs #Leave me alone #After the function runs Yes, that's all, it's that simple. @decorator is just a shortcut to: another_stand_alone_function = my_shiny_new_decorator ( another_stand_alone_function ) Decorators are just a pythonic variant of the decorator design pattern . There are several classic design patterns embedded in Python to ease development, like iterators. Of course, you can cumulate decorators: def bread(func): def wrapper(): print \" < /''''''\\>\" func() print \" < \\______/>\" return wrapper def ingredients(func): def wrapper(): print \"#tomatoes#\" func() print \"~salad~\" return wrapper def sandwich(food=\"--ham--\"): print food sandwich() #outputs: --ham-- sandwich = bread(ingredients(sandwich)) sandwich() #outputs: # < /''''''\\> # #tomatoes# # --ham-- # ~salad~ # < \\______/> Using the Python decorator syntax: @bread @ingredients def sandwich ( food = \"--ham--\" ): print food sandwich () #outputs: #</''''''\\> # #tomatoes# # --ham-- # ~salad~ #<\\______/> The order you set the decorators MATTERS: @ingredients @bread def strange_sandwich ( food = \"--ham--\" ): print food strange_sandwich () #outputs: ##tomatoes# #</''''''\\> # --ham-- #<\\______/> # ~salad~ Eventually answering the question As a conclusion, you can easily see how to answer the question: # The decorator to make it bold def makebold(fn): # The new function the decorator returns def wrapper(): # Insertion of some code before and after return \" <b> \" + fn() + \" </b> \" return wrapper # The decorator to make it italic def makeitalic(fn): # The new function the decorator returns def wrapper(): # Insertion of some code before and after return \" <i> \" + fn() + \" </i> \" return wrapper @makebold @makeitalic def say(): return \"hello\" print say() #outputs: <b><i> hello </i></b> # This is the exact equivalent to def say(): return \"hello\" say = makebold(makeitalic(say)) print say() #outputs: <b><i> hello </i></b> You can now just leave happy, or burn your brain a little bit more and see advanced uses of decorators. Passing arguments to the decorated function # It's not black magic, you just have to let the wrapper # pass the argument: def a_decorator_passing_arguments ( function_to_decorate ): def a_wrapper_accepting_arguments ( arg1 , arg2 ): print \"I got args! Look:\" , arg1 , arg2 function_to_decorate ( arg1 , arg2 ) return a_wrapper_accepting_arguments # Since when you are calling the function returned by the decorator, you are # calling the wrapper, passing arguments to the wrapper will let it pass them to # the decorated function @a_decorator_passing_arguments def print_full_name ( first_name , last_name ): print \"My name is\" , first_name , last_name print_full_name ( \"Peter\" , \"Venkman\" ) # outputs: #I got args! Look: Peter Venkman #My name is Peter Venkman Decorating methods What's great with Python is that methods and functions are really the same, except methods expect their first parameter to be a reference to the current object ( self ). It means you can build a decorator for methods the same way, just remember to take self in consideration: def method_friendly_decorator ( method_to_decorate ) : def wrapper ( self , lie ) : lie = lie - 3 # very friendly , decrease age even more :- ) return method_to_decorate ( self , lie ) return wrapper class Lucy ( object ) : def __ init__ ( self ) : self . age = 32 @method_friendly_decorator def sayYourAge ( self , lie ) : print \"I am %s, what did you think?\" % (self.age + lie) l = Lucy () l . sayYourAge ( - 3 ) # outputs : I am 26 , what did you think? Of course, if you make a very general decorator and want to apply it to any function or method, no matter its arguments, then just use *args, **kwargs : def a_decorator_passing_arbitrary_arguments ( function_to_decorate ): # The wrapper accepts any arguments def a_wrapper_accepting_arbitrary_arguments ( * args , ** kwargs ): print \"Do I have args?:\" print args print kwargs # Then you unpack the arguments, here *args, **kwargs # If you are not familiar with unpacking, check: # http://www.saltycrane.com/blog/2008/01/how-to-use-args-and-kwargs-in-python/ function_to_decorate ( * args , ** kwargs ) return a_wrapper_accepting_arbitrary_arguments @a_decorator_passing_arbitrary_arguments def function_with_no_argument (): print \"Python is cool, no argument here.\" function_with_no_argument () #outputs #Do I have args?: #() #{} #Python is cool, no argument here. @a_decorator_passing_arbitrary_arguments def function_with_arguments ( a , b , c ): print a , b , c function_with_arguments ( 1 , 2 , 3 ) #outputs #Do I have args?: #(1, 2, 3) #{} #1 2 3 @a_decorator_passing_arbitrary_arguments def function_with_named_arguments ( a , b , c , platypus = \"Why not ?\" ): print \"Do %s, %s and %s like platypus? %s\" % ( a , b , c , platypus ) function_with_named_arguments ( \"Bill\" , \"Linus\" , \"Steve\" , platypus = \"Indeed!\" ) #outputs #Do I have args ? : #('Bill', 'Linus', 'Steve') #{'platypus': 'Indeed!'} #Do Bill, Linus and Steve like platypus? Indeed! class Mary ( object ): def __init__ ( self ): self . age = 31 @a_decorator_passing_arbitrary_arguments def sayYourAge ( self , lie =- 3 ): # You can now add a default value print \"I am %s, what did you think ?\" % ( self . age + lie ) m = Mary () m . sayYourAge () #outputs # Do I have args?: #(<__main__.Mary object at 0xb7d303ac>,) #{} #I am 28, what did you think? Passing arguments to the decorator Great, now what would you say about passing arguments to the decorator itself? Well this is a bit twisted because a decorator must accept a function as an argument and therefore, you cannot pass the decorated function arguments directly to the decorator. Before rushing to the solution, let's write a little reminder: # Decorators are ORDINARY functions def my_decorator ( func ) : print \"I am a ordinary function\" def wrapper () : print \"I am function returned by the decorator\" func () return wrapper # Therefore, you can call it without any \"@\" def lazy_function () : print \"zzzzzzzz\" decorated_function = my_decorator ( lazy_function ) #outputs: I am a ordinary function # It outputs \"I am a ordinary function\", because that's just what you do: # calling a function. Nothing magic. @ my_decorator def lazy_function () : print \"zzzzzzzz\" #outputs: I am a ordinary function It's exactly the same. \" my_decorator \" is called. So when you @my_decorator , you are telling Python to call the function 'labeled by the variable \" my_decorator \"'. It's important, because the label you give can point directly to the decorator... or not! Let's start to be evil! def decorator_maker (): print \"I make decorators! I am executed only once: \" + \"when you make me create a decorator.\" def my_decorator ( func ): print \"I am a decorator! I am executed only when you decorate a function.\" def wrapped (): print ( \"I am the wrapper around the decorated function. \" \"I am called when you call the decorated function. \" \"As the wrapper, I return the RESULT of the decorated function.\" ) return func () print \"As the decorator, I return the wrapped function.\" return wrapped print \"As a decorator maker, I return a decorator\" return my_decorator # Let's create a decorator. It's just a new function after all. new_decorator = decorator_maker () #outputs: #I make decorators! I am executed only once: when you make me create a decorator. #As a decorator maker, I return a decorator # Then we decorate the function def decorated_function (): print \"I am the decorated function.\" decorated_function = new_decorator ( decorated_function ) #outputs: #I am a decorator! I am executed only when you decorate a function. #As the decorator, I return the wrapped function # Let's call the function: decorated_function () #outputs: #I am the wrapper around the decorated function. I am called when you call the decorated function. #As the wrapper, I return the RESULT of the decorated function. #I am the decorated function. No surprise here. Let's do EXACTLY the same thing, but skipping intermediate variables: def decorated_function (): print \"I am the decorated function.\" decorated_function = decorator_maker ()( decorated_function ) #outputs: #I make decorators! I am executed only once: when you make me create a decorator. #As a decorator maker, I return a decorator #I am a decorator! I am executed only when you decorate a function. #As the decorator, I return the wrapped function. # Finally: decorated_function () #outputs: #I am the wrapper around the decorated function. I am called when you call the decorated function. #As the wrapper, I return the RESULT of the decorated function. #I am the decorated function. Let's make it AGAIN, even shorter: @decorator_maker () def decorated_function (): print \"I am the decorated function.\" #outputs: #I make decorators! I am executed only once: when you make me create a decorator. #As a decorator maker, I return a decorator #I am a decorator! I am executed only when you decorate a function. #As the decorator, I return the wrapped function. #Eventually: decorated_function () #outputs: #I am the wrapper around the decorated function. I am called when you call the decorated function. #As the wrapper, I return the RESULT of the decorated function. #I am the decorated function. Hey, did you see that? We used a function call with the \" @ \" syntax :-) So back to decorators with arguments. If we can use functions to generate the decorator on the fly, we can pass arguments to that function, right? def decorator_maker_with_arguments ( decorator_arg1 , decorator_arg2 ): print \"I make decorators! And I accept arguments:\" , decorator_arg1 , decorator_arg2 def my_decorator ( func ): # The ability to pass arguments here is a gift from closures. # If you are not comfortable with closures, you can assume it's ok, # or read: http://stackoverflow.com/questions/13857/can-you-explain-closures-as-they-relate-to-python print \"I am the decorator. Somehow you passed me arguments:\" , decorator_arg1 , decorator_arg2 # Don't confuse decorator arguments and function arguments! def wrapped ( function_arg1 , function_arg2 ) : print ( \"I am the wrapper around the decorated function.\\n\" \"I can access all the variables\\n\" \"\\t- from the decorator: {0} {1}\\n\" \"\\t- from the function call: {2} {3}\\n\" \"Then I can pass them to the decorated function\" . format ( decorator_arg1 , decorator_arg2 , function_arg1 , function_arg2 )) return func ( function_arg1 , function_arg2 ) return wrapped return my_decorator @decorator_maker_with_arguments ( \"Leonard\" , \"Sheldon\" ) def decorated_function_with_arguments ( function_arg1 , function_arg2 ): print ( \"I am the decorated function and only knows about my arguments: {0}\" \" {1}\" . format ( function_arg1 , function_arg2 )) decorated_function_with_arguments ( \"Rajesh\" , \"Howard\" ) #outputs: #I make decorators! And I accept arguments: Leonard Sheldon #I am the decorator. Somehow you passed me arguments: Leonard Sheldon #I am the wrapper around the decorated function. #I can access all the variables # - from the decorator: Leonard Sheldon # - from the function call: Rajesh Howard #Then I can pass them to the decorated function #I am the decorated function and only knows about my arguments: Rajesh Howard Here it is, a decorator with arguments. Arguments can be set as variable: c1 = \"Penny\" c2 = \"Leslie\" @decorator_maker_with_arguments ( \"Leonard\" , c1 ) def decorated_function_with_arguments ( function_arg1 , function_arg2 ): print ( \"I am the decorated function and only knows about my arguments:\" \" {0} {1}\" . format ( function_arg1 , function_arg2 )) decorated_function_with_arguments ( c2 , \"Howard\" ) #outputs: #I make decorators! And I accept arguments: Leonard Penny #I am the decorator. Somehow you passed me arguments: Leonard Penny #I am the wrapper around the decorated function. #I can access all the variables # - from the decorator: Leonard Penny # - from the function call: Leslie Howard #Then I can pass them to the decorated function #I am the decorated function and only knows about my arguments: Leslie Howard As you can see, you can pass arguments to the decorator like any function using this trick. You can even use *args, **kwargs if you wish. But remember decorators are called only once . Just when Python imports the script. You can't dynamically set the arguments afterwards. When you do \"import x\", the function is already decorated , so you can't change anything. Let's practice: a decorator to decorate a decorator OK, as a bonus, I'll give you a snippet to make any decorator accept generically any argument. After all, in order to accept arguments, we created our decorator using another function. We wrapped the decorator. Anything else we saw recently that wrapped function? Oh yes, decorators! Let's have some fun and write a decorator for the decorators: def decorator_with_args ( decorator_to_enhance ): \"\"\" This function is supposed to be used as a decorator. It must decorate an other function, that is intended to be used as a decorator. Take a cup of coffee. It will allow any decorator to accept an arbitrary number of arguments, saving you the headache to remember how to do that every time. \"\"\" # We use the same trick we did to pass arguments def decorator_maker ( * args , ** kwargs ): # We create on the fly a decorator that accepts only a function # but keeps the passed arguments from the maker. def decorator_wrapper ( func ): # We return the result of the original decorator, which, after all, # IS JUST AN ORDINARY FUNCTION (which returns a function). # Only pitfall: the decorator must have this specific signature or it won't work: return decorator_to_enhance ( func , * args , ** kwargs ) return decorator_wrapper return decorator_maker It can be used as follows: # You create the function you will use as a decorator. And stick a decorator on it :-) # Don ' t forget , the signature is \"decorator(func, *args, **kwargs)\" @decorator_with_args def decorated_decorator ( func , * args , **kwargs ) : def wrapper ( function_arg1 , function_arg2 ) : print \"Decorated with\" , args , kwargs return func ( function_arg1 , function_arg2 ) return wrapper # Then you decorate the functions you wish with your brand new decorated decorator . @ decorated_decorator ( 42 , 404 , 1024 ) def decorated_function ( function_arg1 , function_arg2 ) : print \"Hello\" , function_arg1 , function_arg2 decorated_function ( \"Universe and\" , \"everything\" ) # outputs : # Decorated with ( 42 , 404 , 1024 ) {} # Hello Universe and everything # Whoooot ! I know, the last time you had this feeling, it was after listening a guy saying: \"before understanding recursion, you must first understand recursion\". But now, don't you feel good about mastering this? Best practices while using decorators They are new as of Python 2.4, so be sure that's what your code is running on. Decorators slow down the function call. Keep that in mind. You can not un-decorate a function. There are hacks to create decorators that can be removed but nobody uses them. So once a function is decorated, it's done. For all the code . Decorators wrap functions, which can make them hard to debug. Python 2.5 solves this last issue by providing the functools module including functools.wraps that copies the name, module and docstring of any wrapped function to it's wrapper. Fun fact, functools.wraps is a decorator :-) # For debugging, the stacktrace prints you the function __name__ def foo (): print \"foo\" print foo . __name__ #outputs: foo # With a decorator, it gets messy def bar ( func ): def wrapper (): print \"bar\" return func () return wrapper @bar def foo (): print \"foo\" print foo . __name__ #outputs: wrapper # \"functools\" can help for that import functools def bar ( func ): # We say that \"wrapper\", is wrapping \"func\" # and the magic begins @functools . wraps ( func ) def wrapper (): print \"bar\" return func () return wrapper @bar def foo (): print \"foo\" print foo . __name__ #outputs: foo How can the decorators be useful? Now the big question: what can I use decorators for? Seem cool and powerful, but a practical example would be great. Well, there are 1000 possibilities. Classic uses are extending a function behavior from an external lib (you can't modify it) or for a debug purpose (you don't want to modify it because it's temporary). You can use them to extends several functions with the same code without rewriting it every time, for DRY's sake. E.g.: def benchmark ( func ): \"\"\" A decorator that prints the time a function takes to execute. \"\"\" import time def wrapper ( * args , ** kwargs ): t = time . clock () res = func ( * args , ** kwargs ) print func . __name__ , time . clock () - t return res return wrapper def logging ( func ): \"\"\" A decorator that logs the activity of the script. (it actually just prints it, but it could be logging!) \"\"\" def wrapper ( * args , ** kwargs ): res = func ( * args , ** kwargs ) print func . __name__ , args , kwargs return res return wrapper def counter ( func ): \"\"\" A decorator that counts and prints the number of times a function has been executed \"\"\" def wrapper ( * args , ** kwargs ): wrapper . count = wrapper . count + 1 res = func ( * args , ** kwargs ) print \"{0} has been used: {1}x\" . format ( func . __name__ , wrapper . count ) return res wrapper . count = 0 return wrapper @counter @benchmark @logging def reverse_string ( string ): return str ( reversed ( string )) print reverse_string ( \"Able was I ere I saw Elba\" ) print reverse_string ( \"A man, a plan, a canoe, pasta, heros, rajahs, a coloratura, maps, snipe, percale, macaroni, a gag, a banana bag, a tan, a tag, a banana bag again (or a camel), a crepe, pins, Spam, a rut, a Rolo, cash, a jar, sore hats, a peon, a canal: Panama!\" ) #outputs: #reverse_string ('Able was I ere I saw Elba',) {} #wrapper 0.0 #wrapper has been used: 1x #ablE was I ere I saw elbA #reverse_string ('A man, a plan, a canoe, pasta, heros, rajahs, a coloratura, maps, snipe, percale, macaroni, a gag, a banana bag, a tan, a tag, a banana bag again (or a camel), a crepe, pins, Spam, a rut, a Rolo, cash, a jar, sore hats, a peon, a canal: Panama!',) {} #wrapper 0.0 #wrapper has been used: 2x #!amanaP :lanac a ,noep a ,stah eros ,raj a ,hsac ,oloR a ,tur a ,mapS ,snip ,eperc a ,)lemac a ro( niaga gab ananab a ,gat a ,nat a ,gab ananab a ,gag a ,inoracam ,elacrep ,epins ,spam ,arutaroloc a ,shajar ,soreh ,atsap ,eonac a ,nalp a ,nam A Of course the good thing with decorators is that you can use them right away on almost anything without rewriting. DRY, I said: @counter @benchmark @logging def get_random_futurama_quote (): import httplib conn = httplib . HTTPConnection ( \"slashdot.org:80\" ) conn . request ( \"HEAD\" , \"/index.html\" ) for key , value in conn . getresponse () . getheaders (): if key . startswith ( \"x-b\" ) or key . startswith ( \"x-f\" ): return value return \"No, I'm ... doesn't!\" print get_random_futurama_quote () print get_random_futurama_quote () #outputs: #get_random_futurama_quote () {} #wrapper 0.02 #wrapper has been used: 1x #The laws of science be a harsh mistress. #get_random_futurama_quote () {} #wrapper 0.01 #wrapper has been used: 2x #Curse you, merciful Poseidon! Python itself provides several decorators: property , staticmethod , etc. Django use decorators to manage caching and view permissions. Twisted to fake inlining asynchronous functions calls. This really is a large playground. Source: Stackoverflow","tags":"help, python, stackoverflow, tutorial","loc":"http://yasoob.github.io/blog/all-about-decorators-in-python","title":"All About Decorators in Python"},{"text":"Hi there folks. Again welcome to yet another useful tutorial. This is again a stackoverflow answer. This one is related to the Python yield keyword. It explains you what yield, generators and iterables are. So without wasting any time lets continue with the answer. To understand what yield does, you must understand what generators are. And before generators come iterables. Iterables When you create a list, you can read its items one by one, and it's called iteration: >>> mylist = [ 1 , 2 , 3 ] >>> for i in mylist: ... print ( i ) 1 2 3 Mylist is an iterable. When you use a list comprehension, you create a list, and so an iterable: >>> mylist = [ x * x for x in range ( 3 )] >>> for i in mylist: ... print ( i ) 0 1 4 Everything you can use \"for... in...\" on is an iterable: lists, strings, files... These iterables are handy because you can read them as much as you wish, but you store all the values in memory and it's not always what you want when you have a lot of values. Generators Generators are iterators, but you can only iterate over them once . It's because they do not store all the values in memory, they generate the values on the fly : >>> mygenerator = ( x * x for x in range ( 3 )) >>> for i in mygenerator: ... print ( i ) 0 1 4 It is just the same except you used () instead of [] . BUT, you can not perform for i in mygenerator a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one. Yield Yield is a keyword that is used like return , except the function will return a generator. >>> def createGenerator (): ... mylist = range ( 3 ) ... for i in mylist: ... yield i * i ... >>> mygenerator = createGenerator () # create a generator >>> print ( mygenerator ) # mygenerator is an object! < generator object createGenerator at 0xb7555c34 > >>> for i in mygenerator: ... print ( i ) 0 1 4 Here it's a useless example, but it's handy when you know your function will return a huge set of values that you will only need to read once. To master yield , you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is a bit tricky :-) Then, your code will be run each time the for uses the generator. Now the hard part: The first time the for calls the generator object created from your function, it will run the code in your function from the beginning until it hits yield , then it'll return the first value of the loop. Then, each other call will run the loop you have written in the function one more time, and return the next value, until there is no value to return. The generator is considered empty once the function runs but does not hit yield anymore. It can be because the loop had come to an end, or because you do not satisfy a \"if/else\" anymore. Your code explained Generator: # Here you create the method of the node object that will return the generator def node . _get_child_candidates ( self , distance , min_dist , max_dist ): # Here is the code that will be called each time you use the generator object: # If there is still a child of the node object on its left # AND if distance is ok, return the next child if self . _leftchild and distance - max_dist < self . _median: yield self . _leftchild # If there is still a child of the node object on its right # AND if distance is ok, return the next child if self . _rightchild and distance + max_dist >= self . _median: yield self . _rightchild # If the function arrives here, the generator will be considered empty # there is no more than two values: the left and the right children Caller: # Create an empty list and a list with the current object reference result , candidates = list (), [ self ] # Loop on candidates (they contain only one element at the beginning) while candidates: # Get the last candidate and remove it from the list node = candidates . pop () # Get the distance between obj and the candidate distance = node . _get_dist ( obj ) # If distance is ok, then you can fill the result if distance <= max_dist and distance >= min_dist: result . extend ( node . _values ) # Add the children of the candidate in the candidates list # so the loop will keep running until it will have looked # at all the children of the children of the children, etc. of the candidate candidates . extend ( node . _get_child_candidates ( distance , min_dist , max_dist )) return result This code contains several smart parts: The loop iterates on a list but the list expands while the loop is being iterated :-) It's a concise way to go through all these nested data even if it's a bit dangerous since you can end up with an infinite loop. In this case, candidates.extend(node._get_child_candidates(distance, min_dist, max_dist)) exhausts all the values of the generator, but while keeps creating new generator objects which will produce different values from the previous ones since it's not applied on the same node. The extend() method is a list object method that expects an iterable and adds its values to the list. Usually we pass a list to it: >>> a = [ 1 , 2 ] >>> b = [ 3 , 4 ] >>> a . extend ( b ) >>> print ( a ) [ 1 , 2 , 3 , 4 ] But in your code it gets a generator, which is good because: You don't need to read the values twice. You can have a lot of children and you don't want them all stored in memory. And it works because Python does not care if the argument of a method is a list or not. Python expects iterables so it will work with strings, lists, tuples and generators! This is called duck typing and is one of the reason why Python is so cool. But this is another story, for another question... You can stop here, or read a little bit to see a advanced use of generator: Controlling a generator exhaustion >>> class Bank (): # let's create a bank, building ATMs ... crisis = False ... def create_atm ( self ): ... while not self . crisis: ... yield \"$100\" >>> hsbc = Bank () # when everything's ok the ATM gives you as much as you want >>> corner_street_atm = hsbc . create_atm () >>> print ( corner_street_atm . next ()) $100 >>> print ( corner_street_atm . next ()) $100 >>> print ([ corner_street_atm . next () for cash in range ( 5 )]) [ '$100' , '$100' , '$100' , '$100' , '$100' ] >>> hsbc . crisis = True # crisis is coming, no more money! >>> print ( corner_street_atm . next ()) < type 'exceptions.StopIteration' > >>> wall_street_atm = hsbc . create_atm () # it's even true for new ATMs >>> print ( wall_street_atm . next ()) < type 'exceptions.StopIteration' > >>> hsbc . crisis = False # trouble is, even post-crisis the ATM remains empty >>> print ( corner_street_atm . next ()) < type 'exceptions.StopIteration' > >>> brand_new_atm = hsbc . create_atm () # build a new one to get back in business >>> for cash in brand_new_atm: ... print cash $100 $100 $100 $100 $100 $100 $100 $100 $100 ... It can be useful for various things like controlling access to a resource. Itertools, your best friend The itertools module contains special functions to manipulate iterables. Ever wish to duplicate a generator? Chain two generators? Group values in a nested list with a one liner? Map / Zip without creating another list? Then just import itertools . An example? Let's see the possible orders of arrival for a 4 horse race: >>> horses = [ 1 , 2 , 3 , 4 ] >>> races = itertools . permutations ( horses ) >>> print ( races ) < itertools . permutations object at 0xb754f1dc > >>> print ( list ( itertools . permutations ( horses ))) [( 1 , 2 , 3 , 4 ), ( 1 , 2 , 4 , 3 ), ( 1 , 3 , 2 , 4 ), ( 1 , 3 , 4 , 2 ), ( 1 , 4 , 2 , 3 ), ( 1 , 4 , 3 , 2 ), ( 2 , 1 , 3 , 4 ), ( 2 , 1 , 4 , 3 ), ( 2 , 3 , 1 , 4 ), ( 2 , 3 , 4 , 1 ), ( 2 , 4 , 1 , 3 ), ( 2 , 4 , 3 , 1 ), ( 3 , 1 , 2 , 4 ), ( 3 , 1 , 4 , 2 ), ( 3 , 2 , 1 , 4 ), ( 3 , 2 , 4 , 1 ), ( 3 , 4 , 1 , 2 ), ( 3 , 4 , 2 , 1 ), ( 4 , 1 , 2 , 3 ), ( 4 , 1 , 3 , 2 ), ( 4 , 2 , 1 , 3 ), ( 4 , 2 , 3 , 1 ), ( 4 , 3 , 1 , 2 ), ( 4 , 3 , 2 , 1 )] Understanding the inner mechanisms of iteration Iteration is a process implying iterables (implementing the __iter__() method) and iterators (implementing the __next__() method). Iterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables. More about it in this article about how does the for loop work . source = Stackoverflow question","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/the-python-yield-keyword-explained","title":"The Python yield keyword explained"},{"text":"Hi there everyone. We all have experienced that time when we say \"what was that function called ?\", \"was it called get_attr() or post_attr() ?\" Recently I came across a wonderful Python Quick Reference for version 2.7 . It contains some useful information. I hope you will find it very useful just like me. So here is the link. Go forward and get it. You will not regret it. Do tell me in the comments that whether you like it or not or do you want something to be added to it. I would really love to hear your response. Let me make it clear as well that I am not the author of this great reference guide. Download Python Quick Reference v2.7 source: Python Quick Ref","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/python-quick-ref-v2-7","title":"Python Quick Reference v2.7"},{"text":"Hi there pythonistas. Recently I was searching about metaclasses in Python and came across a very good explanation about metaclasses on stackoverflow. I found the answer really helpful so I think you should read it as well. Who knows when you might find something useful in there. Disclaimer: very long post. Classes as objects Before understanding metaclasses, you need to master classes in Python. And Python has a very peculiar idea of what classes are, borrowed from the Smalltalk language. In most languages, classes are just pieces of code that describe how to produce an object. That's kinda true in Python too: >>> class ObjectCreator ( object ): ... pass ... >>> my_object = ObjectCreator () >>> print ( my_object ) < __main__ . ObjectCreator object at 0x8974f2c > But classes are more than that in Python. Classes are objects too. Yes, objects. As soon as you use the keyword class , Python executes it and creates an OBJECT. The instruction >>> class ObjectCreator ( object ): ... pass ... creates in memory an object with the name ObjectCreator. This object (the class) is itself capable of creating objects (the instances), and this is why it's a class . But still, it's an object, and therefore: you can assign it to a variable you can copy it you can add attributes to it you can pass it as a function parameter e.g.: >>> print ( ObjectCreator ) # you can print a class because it's an object < class '__main__.ObjectCreator' > >>> def echo ( o ): ... print ( o ) ... >>> echo ( ObjectCreator ) # you can pass a class as a parameter < class '__main__.ObjectCreator' > >>> print ( hasattr ( ObjectCreator , 'new_attribute' )) False >>> ObjectCreator . new_attribute = 'foo' # you can add attributes to a class >>> print ( hasattr ( ObjectCreator , 'new_attribute' )) True >>> print ( ObjectCreator . new_attribute ) foo >>> ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable >>> print ( ObjectCreatorMirror . new_attribute ) foo >>> print ( ObjectCreatorMirror ()) < __main__ . ObjectCreator object at 0x8997b4c > Creating classes dynamically Since classes are objects, you can create them on the fly, like any object. First, you can create a class in a function using class : >>> def choose_class ( name ): ... if name == 'foo' : ... class Foo ( object ): ... pass ... return Foo # return the class, not an instance ... else : ... class Bar ( object ): ... pass ... return Bar ... >>> MyClass = choose_class ( 'foo' ) >>> print ( MyClass ) # the function returns a class, not an instance < class '__main__.Foo' > >>> print ( MyClass ()) # you can create an object from this class < __main__ . Foo object at 0x89c6d4c > But it's not so dynamic, since you still have to write the whole class yourself. Since classes are objects, they must be generated by something. When you use the class keyword, Python creates this object automatically. But as with most things in Python, it gives you a way to do it manually. Remember the function type ? The good old function that lets you know what type an object is: >>> print ( type ( 1 )) < type 'int' > >>> print ( type ( \"1\" )) < type 'str' > >>> print ( type ( ObjectCreator )) < type 'type' > >>> print ( type ( ObjectCreator ())) < class '__main__.ObjectCreator' > Well, type has a completely different ability, it can also create classes on the fly. type can take the description of a class as parameters, and return a class. (I know, it's silly that the same function can have two completely different uses according to the parameters you pass to it. It's an issue due to backwards compatibility in Python) type works this way: type ( name of the class , tuple of the parent class ( for inheritance , can be empty ), dictionary containing attributes names and values ) e.g.: >>> class MyShinyClass ( object ): ... pass can be created manually this way: >>> MyShinyClass = type ( 'MyShinyClass' , (), {}) # returns a class object >>> print ( MyShinyClass ) < class '__main__.MyShinyClass' > >>> print ( MyShinyClass ()) # create an instance with the class < __main__ . MyShinyClass object at 0x8997cec > You'll notice that we use \"MyShinyClass\" as the name of the class and as the variable to hold the class reference. They can be different, but there is no reason to complicate things. type accepts a dictionary to define the attributes of the class. So: >>> class Foo ( object ): ... bar = True Can be translated to: >>> Foo = type ( 'Foo' , (), { 'bar' : True }) And used as a normal class: >>> print ( Foo ) < class '__main__.Foo' > >>> print ( Foo . bar ) True >>> f = Foo () >>> print ( f ) < __main__ . Foo object at 0x8a9b84c > >>> print ( f . bar ) True And of course, you can inherit from it, so: >>> class FooChild ( Foo ): ... pass would be: >>> FooChild = type ( 'FooChild' , ( Foo ,), {}) >>> print ( FooChild ) < class '__main__.FooChild' > >>> print ( FooChild . bar ) # bar is inherited from Foo True Eventually you'll want to add methods to your class. Just define a function with the proper signature and assign it as an attribute. >>> def echo_bar ( self ): ... print ( self . bar ) ... >>> FooChild = type ( 'FooChild' , ( Foo ,), { 'echo_bar' : echo_bar }) >>> hasattr ( Foo , 'echo_bar' ) False >>> hasattr ( FooChild , 'echo_bar' ) True >>> my_foo = FooChild () >>> my_foo . echo_bar () True You see where we are going: in Python, classes are objects, and you can create a class on the fly, dynamically. This is what Python does when you use the keyword class , and it does so by using a metaclass. What are metaclasses (finally) Metaclasses are the 'stuff' that creates classes. You define classes in order to create objects, right? But we learned that Python classes are objects. Well, metaclasses are what create these objects. They are the classes' classes, you can picture them this way: MyClass = MetaClass () MyObject = MyClass () You've seen that type lets you do something like this: MyClass = type ( 'MyClass' , (), {}) It's because the function type is in fact a metaclass. type is the metaclass Python uses to create all classes behind the scenes. Now you wonder why the heck is it written in lowercase, and not Type ? Well, I guess it's a matter of consistency with str , the class that creates strings objects, and int the class that creates integer objects. type is just the class that creates class objects. You see that by checking the __class__ attribute. Everything, and I mean everything, is an object in Python. That includes ints, strings, functions and classes. All of them are objects. And all of them have been created from a class: >>> age = 35 >>> age . __class__ < type 'int' > >>> name = 'bob' >>> name . __class__ < type 'str' > >>> def foo (): pass >>> foo . __class__ < type 'function' > >>> class Bar ( object ): pass >>> b = Bar () >>> b . __class__ < class '__main__.Bar' > Now, what is the __class__ of any __class__ ? >>> age . __class__ . __class__ < type 'type' > >>> name . __class__ . __class__ < type 'type' > >>> foo . __class__ . __class__ < type 'type' > >>> b . __class__ . __class__ < type 'type' > So, a metaclass is just the stuff that creates class objects. You can call it a 'class factory' if you wish. type is the built-in metaclass Python uses, but of course, you can create your own metaclass. The __metaclass__ attribute You can add a __metaclass__ attribute when you write a class: class Foo ( object ): __metaclass__ = something ... [ ... ] If you do so, Python will use the metaclass to create the class Foo . Careful, it's tricky. You write class Foo(object) first, but the class object Foo is not created in memory yet. Python will look for __metaclass__ in the class definition. If it finds it, it will use it to create the object class Foo . If it doesn't, it will use type to create the class. Read that several times. When you do: class Foo ( Bar ): pass Python does the following: Is there a __metaclass__ attribute in Foo ? If yes, create in memory a class object (I said a class object, stay with me here), with the name Foo by using what is in __metaclass__ . If Python can't find __metaclass__ , it will look for a __metaclass__ in Bar (the parent class), and try to do the same. If Python can't find __metaclass__ in any parent, it will look for a __metaclass__ at the MODULE level, and try to do the same. Then if it can't find any __metaclass__ at all, it will use type to create the class object. Now the big question is, what can you put in __metaclass__ ? The answer is: something that can create a class. And what can create a class? type , or anything that subclasses or uses it. Custom metaclasses The main purpose of a metaclass is to change the class automatically, when it's created. You usually do this for APIs, where you want to create classes matching the current context. Imagine a stupid example, where you decide that all classes in your module should have their attributes written in uppercase. There are several ways to do this, but one way is to set __metaclass__ at the module level. This way, all classes of this module will be created using this metaclass, and we just have to tell the metaclass to turn all attributes to uppercase. Luckily, __metaclass__ can actually be any callable, it doesn't need to be a formal class (I know, something with 'class' in its name doesn't need to be a class, go figure... but it's helpful). So we will start with a simple example, by using a function. # the metaclass will automatically get passed the same argument # that you usually pass to `type` def upper_attr ( future_class_name , future_class_parents , future_class_attr ): \"\"\" Return a class object, with the list of its attribute turned into uppercase. \"\"\" # pick up any attribute that doesn't start with '__' and uppercase it uppercase_attr = {} for name , val in future_class_attr . items (): if not name . startswith ( '__' ): uppercase_attr [ name . upper ()] = val else : uppercase_attr [ name ] = val # let `type` do the class creation return type ( future_class_name , future_class_parents , uppercase_attr ) __metaclass__ = upper_attr # this will affect all classes in the module class Foo (): # global __metaclass__ won't work with \"object\" though # but we can define __metaclass__ here instead to affect only this class # and this will work with \"object\" children bar = 'bip' print ( hasattr ( Foo , 'bar' )) # Out: False print ( hasattr ( Foo , 'BAR' )) # Out: True f = Foo () print ( f . BAR ) # Out: 'bip' Now, let's do exactly the same, but using a real class for a metaclass: # remember that `type` is actually a class like `str` and `int` # so you can inherit from it class UpperAttrMetaclass ( type ): # __new__ is the method called before __init__ # it's the method that creates the object and returns it # while __init__ just initializes the object passed as parameter # you rarely use __new__, except when you want to control how the object # is created. # here the created object is the class, and we want to customize it # so we override __new__ # you can do some stuff in __init__ too if you wish # some advanced use involves overriding __call__ as well, but we won't # see this def __new__ ( upperattr_metaclass , future_class_name , future_class_parents , future_class_attr ): uppercase_attr = {} for name , val in future_class_attr . items (): if not name . startswith ( '__' ): uppercase_attr [ name . upper ()] = val else : uppercase_attr [ name ] = val return type ( future_class_name , future_class_parents , uppercase_attr ) But this is not really OOP. We call type directly and we don't override call the parent __new__ . Let's do it: class UpperAttrMetaclass ( type ): def __new__ ( upperattr_metaclass , future_class_name , future_class_parents , future_class_attr ): uppercase_attr = {} for name , val in future_class_attr . items (): if not name . startswith ( '__' ): uppercase_attr [ name . upper ()] = val else : uppercase_attr [ name ] = val # reuse the type.__new__ method # this is basic OOP, nothing magic in there return type . __new__ ( upperattr_metaclass , future_class_name , future_class_parents , uppercase_attr ) You may have noticed the extra argument upperattr_metaclass . There is nothing special about it: a method always receives the current instance as first parameter. Just like you have self for ordinary methods. Of course, the names I used here are long for the sake of clarity, but like for self , all the arguments have conventional names. So a real production metaclass would look like this: class UpperAttrMetaclass ( type ): def __new__ ( cls , clsname , bases , dct ): uppercase_attr = {} for name , val in dct . items (): if not name . startswith ( '__' ): uppercase_attr [ name . upper ()] = val else : uppercase_attr [ name ] = val return type . __new__ ( cls , clsname , bases , uppercase_attr ) We can make it even cleaner by using super , which will ease inheritance (because yes, you can have metaclasses, inheriting from metaclasses, inheriting from type): class UpperAttrMetaclass ( type ): def __new__ ( cls , clsname , bases , dct ): uppercase_attr = {} for name , val in dct . items (): if not name . startswith ( '__' ): uppercase_attr [ name . upper ()] = val else : uppercase_attr [ name ] = val return super ( UpperAttrMetaclass , cls ) . __new__ ( cls , clsname , bases , uppercase_attr ) That's it. There is really nothing more about metaclasses. The reason behind the complexity of the code using metaclasses is not because of metaclasses, it's because you usually use metaclasses to do twisted stuff relying on introspection, manipulating inheritance, vars such as __dict__ , etc. Indeed, metaclasses are especially useful to do black magic, and therefore complicated stuff. But by themselves, they are simple: intercept a class creation modify the class return the modified class Why would you use metaclasses classes instead of functions? Since __metaclass__ can accept any callable, why would you use a class since it's obviously more complicated? There are several reasons to do so: The intention is clear. When you read UpperAttrMetaclass(type) , you know what's going to follow You can use OOP. Metaclass can inherit from metaclass, override parent methods. Metaclasses can even use metaclasses. You can structure your code better. You never use metaclasses for something as trivial as the above example. It's usually for something complicated. Having the ability to make several methods and group them in one class is very useful to make the code easier to read. You can hook on __new__ , __init__ and __call__ . Which will allow you to do different stuff. Even if usually you can do it all in __new__ , some people are just more comfortable using __init__ . These are called metaclasses, damn it! It must mean something! Why the hell would you use metaclasses? Now the big question. Why would you use some obscure error prone feature? Well, usually you don't: Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why). Python Guru Tim Peters The main use case for a metaclass is creating an API. A typical example of this is the Django ORM. It allows you to define something like this: class Person ( models . Model ): name = models . CharField ( max_length = 30 ) age = models . IntegerField () But if you do this: guy = Person ( name = 'bob' , age = '35' ) print ( guy . age ) It won't return an IntegerField object. It will return an int , and can even take it directly from the database. This is possible because models.Model defines __metaclass__ and it uses some magic that will turn the Person you just defined with simple statements into a complex hook to a database field. Django makes something complex look simple by exposing a simple API and using metaclasses, recreating code from this API to do the real job behind the scenes. The last word First, you know that classes are objects that can create instances. Well in fact, classes are themselves instances. Of metaclasses. >>> class Foo ( object ): pass >>> id ( Foo ) 142630324 Everything is an object in Python, and they are all either instances of classes or instances of metaclasses. Except for type . type is actually its own metaclass. This is not something you could reproduce in pure Python, and is done by cheating a little bit at the implementation level. Secondly, metaclasses are complicated. You may not want to use them for very simple class alterations. You can change classes by using two different techniques: monkey patching class decorators 99% of the time you need class alteration, you are better off using these. But 99% of the time, you don't need class alteration at all :-) source : What is a metaclass in python (stackoverflow)","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/all-about-the-metaclasses-in-python","title":"All about the metaclasses in Python!"},{"text":"Hi there pythonistas. I hope you are all fine. In this post I am going to teach you how we can make a Reddit + Twitter bot. What this bot will do is that it will copy post titles and url from any subreddit that you want and then it will post them to twitter keeping the 140 characters length in mind. Firstly let me tell you what Reddit is. Reddit is a social link sharing site where good links are upvoted and bad links are down voted. So lets start. Required External Libraries 1. Praw - (Reddidt API wrapper in python) pip install praw 2. Requests - (HTTP library) pip install requests 3. Tweepy - (Twitter python API) pip install tweepy First step - Register Yourself So first of all you will have to register an app on http://dev.twitter.com/apps and after registering copy the access token access token secret consumer key consumer secret You will have to edit the permissions for your app under the settings tab and grant your application read and write permission. So now we are ready to move on. Start by importing So now lets start writing our script. First of all we will have to import the required libraries and set up some basic variables: import praw import json import requests import tweepy import time access_token = 'YOUR ACCESS TOKEN HERE' access_token_secret = 'YOUR ACCESS TOKEN SECRET HERE' consumer_key = 'YOUR CONSUMER KEY HERE' consumer_secret = 'YOUR CONSUMER SECRET HERE' Initiate connection with Reddit Now we have to initiate connection with Reddit. Lets define a function to do just that. def setup_connection_reddit ( subreddit ): print \"[bot] setting up connection with Reddit\" r = praw . Reddit ( 'yasoob_python reddit twitter bot ' 'monitoring %s' % ( subreddit )) subreddit = r . get_subreddit ( subreddit ) return subreddit This method connects with Reddit and gets the subreddit of our choice and then returns that subreddit for us to work further with. Get posts and links from Reddit So now we have to define a function that gets the list of posts and there urls from Reddit for us. So lets just do that as well. def tweet_creator ( subreddit_info ): post_dict = {} post_ids = [] print \"[bot] Getting posts from Reddit\" for submission in subreddit_info . get_hot ( limit = 20 ): # strip_title function is defined later post_dict [ strip_title ( submission . title )] = submission . url post_ids . append ( submission . id ) print \"[bot] Generating short link using goo.gl\" mini_post_dict = {} for post in post_dict: post_title = post post_link = post_dict [ post ] # the shorten function is defined later short_link = shorten ( post_link ) mini_post_dict [ post_title ] = short_link return mini_post_dict , post_ids First of all we are declaring a dictionary to hold the post title and link and after that we are making a list to hold the unique ids of every post that we grab. This is used to track which posts we have already grabbed. After that we are looping over the posts and appending values to the dictionary and the list. If you use twitter very frequently then you know that how disgusting long links look like so in order to tackle that we are using goo.gl to generate short links for us. That's the next thing we have done in the above function. We loop over the post dict and make a short link for every link and append it to a new dictionary which is mini_post_dict . No one likes long links Now lets define a function which will actually shorten the links for us. So here it is: def shorten ( url ): headers = { 'content-type' : 'application/json' } payload = { \"longUrl\" : url } url = \"https://www.googleapis.com/urlshortener/v1/url\" r = requests . post ( url , data = json . dumps ( payload ), headers = headers ) link = json . loads ( r . text )[ 'id' ] return link The above function contains a header and a payload which we are going to send to google and after that google will return the short link in the form of json. Twitter hates more than 140 characters If you use twitter regularly then I am sure that you know that twitter does not like tweets that are more than 140 characters. So in order to tackle that lets define a function that will truncate long tweets to short ones. def strip_title ( title ): if len ( title ) < 94 : return title else : return title [: 93 ] + \"...\" In the above method we will pass a title and the above method will check that whether the title is 93 characters or more. If it is more than 93 characters then it will truncate it and append three dots at its end. Dealing with duplicate posts So now we have started to shape our final script. There is one thing that we have to keep in mind. No one likes duplicate posts so we have to make sure that we do not post same tweets over and over again. In order to tackle this issue we are going to make a file with the name of posted_posts.txt . When ever we grab a post from Reddit we will add it's ID to this file and when posting to twitter we will check whether the post with this ID has already been posted or not. Lets define two more functions. The first one will write the IDs to file and the second one will check whether the post is already posted or not. def add_id_to_file ( id ): with open ( 'posted_posts.txt' , 'a' ) as file: file . write ( str ( id ) + \"\\n\" ) def duplicate_check ( id ): found = 0 with open ( 'posted_posts.txt' , 'r' ) as file: for line in file: if id in line: found = 1 return found Make a function for twitter will ya So now lets make our one of the main function. This function is actually going to post to twitter. def tweeter ( post_dict , post_ids ): auth = tweepy . OAuthHandler ( consumer_key , consumer_secret ) auth . set_access_token ( access_token , access_token_secret ) api = tweepy . API ( auth ) for post , post_id in zip ( post_dict , post_ids ): found = duplicate_check ( post_id ) if found == 0 : print \"[bot] Posting this link on twitter\" print post + \" \" + post_dict [ post ] + \" #Python #reddit #bot\" api . update_status ( post + \" \" + post_dict [ post ] + \" #Python #reddit #bot\" ) add_id_to_file ( post_id ) time . sleep ( 30 ) else : print \"[bot] Already posted\" Firstly we setup connection with twitter by using the credentials we defined in the beginning. After that we loop over the post_dict and post_ids. Then we check for duplicate posting. If it is not previously posted then we post it and add the id of the post in the posted_posts.txt file. After posting we wait for 30 seconds so that we do not spam twitter with tweets. Wheres the main function bud So lets define our last function. This function will co ordinate with all other functions. Here is the code for that last function: def main (): subreddit = setup_connection_reddit ( 'python' ) post_dict , post_ids = tweet_creator ( subreddit ) tweeter ( post_dict , post_ids ) So now we are ready just add this little line at the end as well: if __name__ == '__main__' : main () This checks whether the script is directly executed or is it imported. If it is directly executed only then the main() function is executed. Complete code Here is the complete script: import praw import json import requests import tweepy import time access_token = 'YOUR ACCESS TOKEN HERE' access_token_secret = 'YOUR ACCESS TOKEN SECRET HERE' consumer_key = 'YOUR CONSUMER KEY HERE' consumer_secret = 'YOUR CONSUMER SECRET HERE' def strip_title ( title ): if len ( title ) < 94 : return title else : return title [: 93 ] + \"...\" def tweet_creator ( subreddit_info ): post_dict = {} post_ids = [] print \"[bot] Getting posts from Reddit\" for submission in subreddit_info . get_hot ( limit = 20 ): post_dict [ strip_title ( submission . title )] = submission . url post_ids . append ( submission . id ) print \"[bot] Generating short link using goo.gl\" mini_post_dict = {} for post in post_dict: post_title = post post_link = post_dict [ post ] short_link = shorten ( post_link ) mini_post_dict [ post_title ] = short_link return mini_post_dict , post_ids def setup_connection_reddit ( subreddit ): print \"[bot] setting up connection with Reddit\" r = praw . Reddit ( 'yasoob_python reddit twitter bot ' 'monitoring %s' % ( subreddit )) subreddit = r . get_subreddit ( subreddit ) return subreddit def shorten ( url ): headers = { 'content-type' : 'application/json' } payload = { \"longUrl\" : url } url = \"https://www.googleapis.com/urlshortener/v1/url\" r = requests . post ( url , data = json . dumps ( payload ), headers = headers ) link = json . loads ( r . text )[ 'id' ] return link def duplicate_check ( id ): found = 0 with open ( 'posted_posts.txt' , 'r' ) as file: for line in file: if id in line: found = 1 return found def add_id_to_file ( id ): with open ( 'posted_posts.txt' , 'a' ) as file: file . write ( str ( id ) + \"\\n\" ) def main (): subreddit = setup_connection_reddit ( 'python' ) post_dict , post_ids = tweet_creator ( subreddit ) tweeter ( post_dict , post_ids ) def tweeter ( post_dict , post_ids ): auth = tweepy . OAuthHandler ( consumer_key , consumer_secret ) auth . set_access_token ( access_token , access_token_secret ) api = tweepy . API ( auth ) for post , post_id in zip ( post_dict , post_ids ): found = duplicate_check ( post_id ) if found == 0 : print \"[bot] Posting this link on twitter\" print post + \" \" + post_dict [ post ] + \" #Python #reddit #bot\" api . update_status ( post + \" \" + post_dict [ post ] + \" #Python #reddit #bot\" ) add_id_to_file ( post_id ) time . sleep ( 30 ) else : print \"[bot] Already posted\" if __name__ == '__main__' : main () Save this file with the name of reddit_bot.py and make a file with the name of posted_posts.txt and then execute the python script from the terminal. Your output will look something like this: yasoob @yasoob: ~/ Desktop $ python reddit_bot . py [ bot ] setting up connection with Reddit [ bot ] Getting posts from Reddit [ bot ] Generating short link using goo . gl [ bot ] Posting this link on twitter Miloslav Trma \u010d , - 1 for Structured Logging http: //goo . gl / sF8Xgm #Python #reddit #bot And after some time your posted_posts.txt file will look something like this: 1 mb4y4 1 mb867 1 mb4hl 1 mbh3t 1 mbni0 1 m9bod 1 mbhpt 1 mbhnc 1 mbcp2 1 m9d2t 1 maeio 1 m9bi5 1 m8tgr 1 m86e4 1 ma5r5 1 m8fud 1 mdh1t 1 mbst4 Goodbye I hope you enjoyed today's post as much as I enjoyed writing it. I hope to see you in future with some more tutorials. Do follow my blog to give me some support and get regular updates. Goodbye till next time.","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/making-a-reddit-twitter-bot","title":"Making a Reddit + twitter Bot"},{"text":"Hi there pythonistas. You might have noticed that I haven't been active much on this blog. There's a reason for that. I was preparing a PyCon talk. Yes I was really preparing a PyCon talk. The title of the talk is web scraping in python. I have submitted the proposal and now I am waiting for it's acceptance. Do pray for it's acceptance and after that you will be able to see me in Montreal next year ;) . I will keep you updated with the status of my talk that whether it is accepted or not.","tags":"Uncategorized","loc":"http://yasoob.github.io/blog/a-surprize-for-you","title":"A surprize for you"},{"text":"Hi there guys. I hope you are fine. In this post I am going to tell you how you can make a twitter auto favouriter or a.k.a Twitter Bot. I am not going to make a twitter follower as it will get you banned quickly if you use it a lot. This post is going to help you to increase your twitter followers organically. By organically I mean that your followers will be those who really want to listen to your content. You can either use services on the web to increase your followers inorganically or else you can use this script ;) . So what affect does favouriting has on the user who's post you favourite ? When you favourite someone's post he will get curious as who favourited his post. Then he will visit your profile and will follow you if he likes your tweets. So without wasting any time lets begin. The basic mechanism or the way of working of our bot will be that you will give it a keyword like \"python\" and it will search twitter for that keyword. After searching it will start favouriting the tweets from top to bottom. First of all I am going to use \"twitter\" module for this project. There are a lot of twitter related modules on PyPI but this module did the trick so I thought to use it (https://pypi.python.org/pypi/twitter/1.10.0) . Secondly in order to follow along you will have to make a dev account on twitter and register an application with any name (https://dev.twitter.com/apps). Lets begin by importing the twitter module and initiating connection with twitter using the OAUTH tokens. : from twitter import Twitter , OAuth , TwitterHTTPError OAUTH_TOKEN = 'your oauth token' OAUTH_SECRET = 'your oauth secret' CONSUMER_KEY = 'your consumer key' CONSUMER_SECRET = 'your consumer secret' t = Twitter ( auth = OAuth ( OAUTH_TOKEN , OAUTH_SECRET , CONSUMER_KEY , CONSUMER_SECRET )) So now we have initiated connection with twitter. This is the basic structure of our twitter auto favouriter or a.k.a twitter bot. Now lets move on and define a method which will search for tweets on twitter. def search_tweets ( q , count = 100 ): return t . search . tweets ( q = q , result_type = 'recent' , count = count ) So now we have to define a method which will favourite a tweet: def fav_tweet ( tweet ): try : result = t . favorites . create ( _id = tweet [ 'id' ]) print \"Favorited: %s \" % ( result [ 'text' ]) return result # when you have already favourited a tweet # this error is thrown except TwitterHTTPError as e : print \"Error: \" , e return None So now we have a searching method and a favouriting method. Now tell me what is left ? Obviously the main code that will drive our whole program is left. So here you go: def auto_fav ( q , count = 100 ): result = search_tweets ( q , count ) a = result [ 'statuses' ][ 0 ][ 'user' ][ 'screen_name' ] print a success = 0 for tweet in result [ 'statuses' ]: if fav_tweet ( tweet ) is not None : success += 1 print \"We Favorited a total of %i out of %i tweets\" % ( success , len ( result [ 'statuses' ])) So thats it! We have a complete bot for twitter. This bot currently only favourites tweets based on a keyword. Save this code in a file called twitter_bot.py . Now you can use this bot by typing this in the terminal: $ python >>> import twitter_bot >>> twitter_bot . auto_fav ( '#python' , '20' ) The above command will search for 20 latest tweets containing the hashtag \"python\". After that it will favourite all of those tweets. This bot itself works but I can not gurantee that you will get followers quickly and I can not even gurantee that you will not get banned by twitter for using this a lot. You can use this once in a day. Hopefully I will write a twitter follower bot in the future as well but till then this script is your best bet. This is the complete script: from twitter import Twitter , OAuth , TwitterHTTPError OAUTH_TOKEN = 'your oauth token' OAUTH_SECRET = 'your oauth secret' CONSUMER_KEY = 'your consumer key' CONSUMER_SECRET = 'your consumer secret' t = Twitter ( auth = OAuth ( OAUTH_TOKEN , OAUTH_SECRET , CONSUMER_KEY , CONSUMER_SECRET )) def search_tweets ( q , count = 100 ): return t . search . tweets ( q = q , result_type = 'recent' , count = count ) def fav_tweet ( tweet ): try : result = t . favorites . create ( _id = tweet [ 'id' ]) print \"Favorited: %s \" % ( result [ 'text' ]) return result # when you have already favourited a tweet # this error is thrown except TwitterHTTPError as e : print \"Error: \" , e return None def auto_fav ( q , count = 100 ): result = search_tweets ( q , count ) a = result [ 'statuses' ][ 0 ][ 'user' ][ 'screen_name' ] print a success = 0 for tweet in result [ 'statuses' ]: if fav_tweet ( tweet ) is not None : success += 1 print \"We Favorited a total of %i out of %i tweets\" % ( success , len ( result [ 'statuses' ])) This is the final result of running this sript: yasoob@yasoob:~/Desktop$ python Python 2.7.3 (default, Apr 10 2013, 06:20:15) [GCC 4.6.3] on linux2 Type \"help\", \"copyright\", \"credits\" or \"license\" for more information. >>> import twitter_bot as bot >>> bot.auto_fav('django',1) Favorited: Enjoying @freakboy3742's discussion of cultural naming patterns vis a vis Django users at #DjangoCon We Favorited a total of 1 out of 1 tweets >>> I hope you liked todays post. If you have any questions then feel free to comment below and do follow us if you want to get regular updates from our blog. If you want me to write on one specific topic then do tell it to me in the comments below. Stay tuned for my next post.","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/increase-your-twitter-followers","title":"Making a twitter bot in python"},{"text":"Beginner's Delight Beginner's guide to Python The Hitchhiker's guide to Python Learn Python the hard way Google's Python class Python for Beginners Python tracks at Codecademy Python Monk Best way to learn python Python Tips, tools and resources Python Koans: Learn Python through TDD LearnStreet Python course Beginner projects for a python newbie Learn X in Y minutes - Python Python Practice book Facts and myths about Python names and values [ args and * kwargs in python explained][] Common newbie mistakes in Python- Part 1 Common newbie mistakes in Python- Part 2 PySchools Some cool Python Tricks http://blog.amir.rachum.com/post/30176371115/you-cant-handle-the-truth Better Python APIs Python Gotchas Declaring dependencies in Python Style Guide and Idioms PEP 8 - Style Guide for Python Code Code Like a Pythonista: Idiomatic Python Dictionary Python hash, id and dictionary order Notes on dictionary implementation Decorators Understanding Python Decorators in 12 easy steps Common uses of Python decorators - Quora Common uses of Python decorators - Stack Overflow Python Decorators Decorators I: Introduction to Python Decorators Decorators and annotations The dark side of decorators Thinking out aloud: Python decorators Using Python decorators for registering callbacks Meta-matters: Using decorators for better Python programming Generators Generator Tricks for Systems Programmers Yield Python yield keyword explained Context Managers Understanding Python's with statement Unicode Explain Unicode like I'm five years old Unicode Pain Python encodings and Unicode The Updated Guide to Unicode on Python Metaclasses A primer on python metaclasses What is a metaclass in Python ? Yet Another Python Metaclass tutorial Documentation Documentation is King Make your open source project documentation suck less Sphinx Quick Sphinx documentation for Python Debugging Debugging Tools in Python Testing Nose Introduction Unittest Introduction How To Use Coverage.py With Unittest Profiling A guide to analyzing Python performance Packaging Sharing Your Labor of Love: PyPI Quick And Dirty Open Sourcing a Python Project the Right Way Deployment Fabric Using Fabric in a development environment Web Python and Realtime Web Frameworks Flask Flask Mega Tutorial Series Designing a RESTful API with Python and Flask Building websites in Python with Flask How I structure my Flask apps Designing a RESTful API using Flask-RESTful Rapid Web Prototyping with Lightweight Tools Web2Py Building a Twitter clone in Web2Py Django The official Django documentation Getting started with Django Building a hacker news clone in Django Django Architecture: The good, bad and ugly http://www.lightbird.net/dbe/ http://showmedo.com/videotutorials/series?name=PPN7NA155 Bottle Developing with Bottle- Part 1 API and Web Services How to access various web services in python Scraping Web Scraping with BeautifulSoup Web Scraping 101 with Python [How to automatically search and download torrents with Python and Scrapy][] Mobile Development Kivy Creating an app in Kivy Image Processing http://www.riisen.dk/dop/pil.html WSGI Getting started with WSGI WSGI Basics Databases SQLAlchemy http://www.aosabook.org/en/sqlalchemy.html http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/ Data Mining Linux system mining using Python Data Analysis Pandas A 10 min intro to Pandas http://manishamde.github.io/blog/2013/03/07/pandas-and-python-top-10/ Design Patterns Python Best Practice Patterns by Vladimir Keleshev Concurrency Patterns Wasps Nest Pattern Wasp's Nest: The Read-Copy-Update Pattern In Python Distributed Systems http://architects.dzone.com/articles/gevent-zeromq Python 2 vs 3 Python 2 or 3 ? Porting to Python 3 http://python3porting.com/ Books Free Some free books on Python Invent with Python Think Python Byte of Python Snake Wrangling for Kids [Data Structures and Algorithms with Object-Oriented Design Patterns in Python][] Python programming on wikibooks http://readwrite.com/2011/03/25/python-is-an-increasingly-popu#awesm=\\~oalneJOKY2tp7U Dive Into Python 3 Paid Some list of paid books on python Discussions https://news.ycombinator.com/item?id=3746692 https://news.ycombinator.com/item?id=5944863 http://stackoverflow.com/questions/92230/python-beyond-the-basics http://stackoverflow.com/questions/2573135/python-progression-path-from-apprentice-to-guru http://stackoverflow.com/questions/228181/zen-of-python http://stackoverflow.com/questions/101268/hidden-features-of-python Conferences and events Official PyCon website http://lanyrd.com/topics/python/ http://lanyrd.com/topics/django/ http://python.meetup.com/ http://django.meetup.com/ Videos Guido Van Rossum on the history of Python PyVideo http://lanyrd.com/topics/python/video/ http://showmedo.com/videotutorials/python Talks by Brandon Rhodes Editors and IDE's for Python programming Setting up Sublime Text for Python Development Sublime Text 2 Love by Kenneth Reitz Sublime Text 2 tips for Python and web developers Miscellaneous Good to great Python reads How to Create Barcodes in Your PDFs with Python What are some good Python interview questions ? Python innards introduction source: pycrumbs github [How to automatically search and download torrents with Python and Scrapy]: http://fuzz-box.blogspot.in/2013/03/how-to-automatically-search-download-torrent-python-scrapy.html [Data Structures and Algorithms with Object-Oriented Design Patterns in Python]: http://www.brpreiss.com/books/opus7/html/book.html","tags":"misc","loc":"http://yasoob.github.io/blog/python-resources","title":"Python Resources"},{"text":"Hi there pythonistas! We all know that Sudoku is a great game. Some of us even bet on this game but did you know that you can use python to make a Sudoku solver ? In this post I am going to share with you a Sudoku solver written in python. From now on you will win all Sudoku challenges. However let me tell you that I am not the original writer of this script. I stumbled over this script on stack-overflow. So without wasting any time let me share the script with you: import sys def same_row ( i , j ): return ( i /9 == j/ 9 ) def same_col ( i , j ): return ( i - j ) % 9 == 0 def same_block ( i , j ): return ( i /27 == j/ 27 and i %9 /3 == j%9/ 3 ) def r ( a ): i = a . find ( '0' ) if i == - 1 : sys . exit ( a ) excluded_numbers = set () for j in range ( 81 ): if same_row ( i , j ) or same_col ( i , j ) or same_block ( i , j ): excluded_numbers . add ( a [ j ]) for m in '123456789' : if m not in excluded_numbers: r ( a [: i ] + m + a [ i + 1 :]) if __name__ == '__main__' : if len ( sys . argv ) == 2 and len ( sys . argv [ 1 ]) == 81 : r ( sys . argv [ 1 ]) else : print 'Usage: python sudoku.py puzzle' print ' where puzzle is an 81 character string representing the puzzle read left-to-right, top-to-bottom, and 0 is a blank' Hey there wait! let me share with you a shorter obfuscated version of the same Sudoku solving script. However this short version of Sudoku solver is a lot slower but I think that I should share it with you just to show you that even in python obfuscated code can be written. So here is the shorter obfuscated version: def r ( a ): i = a . find ( '0' ); ~ i or exit ( a );[ m in [( i - j ) %9 * ( i /9^j/ 9 ) * ( i /27^j/ 27 | i %9 /3^j%9/ 3 ) or a [ j ] for j in range ( 81 )] or r ( a [: i ] + m + a [ i + 1 :]) for m in '%d' %5 ** 18 ] from sys import * ; r ( argv [ 1 ]) If you liked this post about Sudoku solver then don't forget to share this on twitter and facebook. If you want to get regular updates then don't forget to follow our blog.","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/sudoku-solver-in-python","title":"Simple Sudoku solver in python"},{"text":"Hi there folks. In this post I am going to teach you how to increase the recursion depth in python. Most of us at some time get this error : RuntimeError : maximum recursion depth exceeded If you want to fix this error just increase the default recursion depth limit but how to do it ? Just import the sys module in your script and in the beginning of the script type this : sys . setrecursionlimit ( 1500 ) This will increase the default limit to 1500. For the record the default limit is 1000. I hope that you found this post useful. Do share this on facebook and twitter and stay tuned for our next post.","tags":"help, python","loc":"http://yasoob.github.io/blog/fixing-error-maximum-recursion-depth-reached","title":"Fixing error - maximum recursion depth reached"},{"text":"Hi there folks. We all know that dropbox is made in python. It's website is made with pyramid and it's desktop applications are made with python. The existing Python bytecode reversing techniques are not enough for reversing hardened applications like Dropbox. Recently I came accross a whitepaper written by Dhiru Kholia and Przemys\u0142aw Wegrzyn. This paper presents new and generic techniques, to reverse engineer frozen Python applications, which are not limited to just the Dropbox world. It describes a method to bypass Dropbox's two factor authentication and hijack Dropbox accounts. Additionally, generic techniques to intercept SSL data using code injection techniques and monkey patching are presented. The methods presented in this whitepaper are not limited to dropbox. You can download this whitepaper from usenix website. Do share your views after reading this whitepaper and don't hesitate following us in order to get regular updates.","tags":"python, whitepaper","loc":"http://yasoob.github.io/blog/looking-inside-dropbox-whitepaper","title":"Looking Inside Dropbox - whitepaper"},{"text":"Hi there folks. I know I have not been active recently and it was because I was not in town. I was in Canada. So this post will be short. In this post i will show you how you can find the version number of any python module. Sometimes you need to find the version number in order to know whether you have the desired version of the module or not. So here is the trick. \\1. First of all launch python: $ python \\2. Import the module which in our case is requests: >>> import requests \\3. Lastly type this: >>> requests . __version__ '1.2.3' So I hope you got the trick. If you have any questions then feel free to post them below and stay tuned for the next post.","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/finding-the-module-version","title":"Finding the module version"},{"text":"This is a very nice post which teaches you some of the basics of PIL (python imaging library) . Make sure that you read it if you are interested in images.","tags":"Uncategorized","loc":"http://yasoob.github.io/blog/extracting-pixel-values-of-an-image-in-python","title":"Extracting pixel values of an image in python"},{"text":"Hi there folks. Some time ago i was very much interested in how music downloaders for various websites worked. In order to learn this i went forward to make my own downloader for ex.fm . Currently ex.fm does not allow users to download songs so this was a very good chance for me to learn song downloading with python. During this process i also came forward to youtube-dl. It is a python script which allows anyone to download music and videos from numerous websites. Now let me tell you how i went forward to make this downloader. First of all I opened ex.fm in chrome and also started chrome web inspector to intercept network calls. Then i clicked on a song for it to play. When the song started to play i saw the XHR requests made by ex.fm . There i saw a request being made to ex.fm/api . This was interesting because most websites use apis to provide info for specific songs. I opened the link and saw that the link returned json data with direct link to the song. That way i came to know the url used by ex.fm to get the direct link to the song. After that I wanted to know how to automate song searching from python. For this i again opened ex.fm and chrome web inspector. This time instead of playing a song i clicked on search and searched for a song. Again a url got my attention. It was http://ex.fm/api/v3/song/search/{search term}?start=0&results=20 . This url also returned json response. After i became aware of these two urls i started to write a python script to automate this. I used only three libraries and all three of them come preinstalled with python (batteries included). These three libraries were urllib2 json sys I won't be explaining the code here instead i will give you the code itself. If you have any questions about the code feel free to comment below. Lastly let me make it clear that this script can be written in better ways but i just made it for learning purposes. [Here is the link][] to the github repository. [Here is the link]: https://github.com/yasoob/ex.fm-dl","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/making-a-song-downloader-with-python","title":"Making a song downloader with python"},{"text":"One of the latest application made with python. Definitely worth checking out. Do share your views in the comments below.","tags":"Uncategorized","loc":"http://yasoob.github.io/blog/software-spotlight-bitmessage","title":"Software Spotlight: Bitmessage"},{"text":"We've already learned about pickle , so why do we need another way to (de)serialize Python objects to(from) disk or a network connection? There are three major reasons to prefer JSON over pickle: When you're unpickling data, you're essentially allowing your data source to execute arbitrary Python commands . If the data is trustworthy (say stored in a sufficiently protected directory), that may not be a problem, but it's often really easy to accidentally leave a file unprotected (or read something from network). In these cases, you want to load data, and not execute potentially malicious Python code! Pickled data is not easy to read, and virtually impossible to write for humans. For example, the pickled version of {\"answer\": [42]} looks like this: ( dp0 S 'answer' p1 ( lp2 I42 as . In contrast, the JSON representation of {\"answer\": [42]} is .... {\"answer\": [42]} . If you can read Python, you can read JSON; since all JSON is valid Python code! Pickle is Python-specific. In fact, by default, the bytes generated by Python 3's pickle cannot be read by a Python 2.x application! JSON can be read by virtually any programming language - just scroll down on the official homepage to see implementations in all major and some minor languages . So how do you get the JSON representation of an object? It's simple, just call [ json.dumps ][]: import json obj = { u \"answer\" : [ 42.2 ], u \"abs\" : 42 } print ( json . dumps ( obj )) # output: {\"answer\": [42.2], \"abs\": 42} Often, you want to write to a file or network stream. In both Python 2.x and 3.x you can call [ dump ][] to do that, but in 3.x the output must be a character stream, whereas 2.x expects a byte stream. Let's look how to load what we wrote. Fittingly, the function to load is called loads (to load from a string) / load (to load from a stream): import json obj_json = u '{\"answer\": [42.2], \"abs\": 42}' obj = json . loads ( obj_json ) print ( repr ( obj )) When the objects we load and store grow larger, we puny humans often need some hints on where a new sub-object starts. To get these, simply pass an indent size, like this: import json obj = { u \"answer\" : [ 42.2 ], u \"abs\" : 42 } print ( json . dumps ( obj , indent = 4 )) Now, the output will be a beautiful { \"abs\" : 42 , \"answer\" : [ 42.2 ] } I often use this indentation feature to debug complex data structures. The price of JSON's interoperability is that we cannot store arbitrary Python objects. In fact, JSON can only store the following objects: character strings numbers booleans ( True / False ) None lists dictionaries with character string keys Every object that's not one of these must be converted - that includes every object of a custom class. Say we have an object alice as follows: class User ( object ): def __init__ ( self , name , password ): self . name = name self . password = password alice = User ( 'Alice A. Adams' , 'secret' ) then converting this object to JSON will fail: >>> import json >>> json . dumps ( alice ) Traceback ( most recent call last ): File \"<stdin>\" , line 1 , in <module> File \"/usr/lib/python3.3/json/__init__.py\" , line 236 , in dumps return _default_encoder . encode ( obj ) File \"/usr/lib/python3.3/json/encoder.py\" , line 191 , in encode chunks = self . iterencode ( o , _one_shot = True ) File \"/usr/lib/python3.3/json/encoder.py\" , line 249 , in iterencode return _iterencode ( o , 0 ) File \"/usr/lib/python3.3/json/encoder.py\" , line 173 , in default raise TypeError ( repr ( o ) + \" is not JSON serializable\" ) TypeError: < __main__ . User object at 0x7f2eccc88150 > is not JSON serializable Fortunately, there is a simple hook for conversion: Simply define a default method.: def jdefault ( o ): return o . __dict__ print ( json . dumps ( alice , default = jdefault )) # outputs: {\"password\": \"secret\", \"name\": \"Alice A. Adams\"} o.__dict__ is a simple catch-all for user-defined objects, but we can also add support for other objects. For example, let's add support for sets by treating them like lists: def jdefault ( o ): if isinstance ( o , set ): return list ( o ) return o . __dict__ pets = set ([ u 'Tiger' , u 'Panther' , u 'Toad' ]) print ( json . dumps ( pets , default = jdefault )) # outputs: [\"Tiger\", \"Panther\", \"Toad\"] For more options and details ( ensure_ascii and sort_keys may be interesting options to set), have a look at the official documentation for JSON . JSON is available by default in Python 2.6 and newer, before that you can use simplejson as a fallback . You might also like : *) The self variable in python explained *) Python socket network programming *) [*args and **kwargs in python explained][]","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/storing-and-loading-data-with-json","title":"Storing and Loading Data with JSON"},{"text":"Hi there fellas. In this post I am going to teach you about the self variable in python. I have seen many beginners struggling to grasp the concept of self variable. If you are one of them then this post is for you. So lets start by making a class involving the self variable. A simple class : So here is our class: class Restaurant ( object ): bankrupt = False def open_branch ( self ): if not self . bankrupt: print ( \"branch opened\" ) First let me explain the above code without the technicalities. First of all we make a class Restaurant. Then we assign it a property \"bankrupt\" which is currently false. After that we assign it a function open_branch which can only occur if \"bankrupt\" is False which means that the Restaurant has some money. Making a resturant: Now that we have made a class for a Restaurant, lets actually make a resturant: x = Restaurant () Now x is a Restaurant which has a property bankrupt and a function open_branch. Now we can access the property bankrupt by typing: x . bankrupt The above command is same as: Restaurant () . bankrupt Now you can see that self refers to the bound variable or object. In the first case it was x because we had assigned the Restaurant class to x whereas in the second case it referred to Restaurant(). Now if we have another Restaurant y, self will know to access the bankrupt value of y and not x. For example check this example: >>> x = Restaurant () >>> x . bankrupt False >>> y = Restaurant () >>> y . bankrupt = True >>> y . bankrupt True >>> x . bankrupt False The first argument of every class method, including __init__, is always a reference to the current instance of the class. By convention, this argument is always named self. In the __init__ method, self refers to the newly created object; in other class methods, it refers to the instance whose method was called. For example the below code is the same as the above code. class Restaurant ( object ): bankrupt = False def open_branch ( this ): if not this . bankrupt: print ( \"branch opened\" ) Free Tip: However self is not a reserved keyword in python it's just a strong convention. Many people say that why do we have to write self ? Why can't we have it set automatically like in Java ? Someone also filed a PEP (improvement suggestion) in which he suggested to remove the explicit assignment of self keyword. However Guido Van Rossum (the maker of python) wrote a blogpost in which he told why explicit self has to stay . I hope you have understood to concept of self. If you have any other questions then feel free to comment. If you liked this post then make sure that you share it on facebook, tweet it on twitter and follow our blog. You might also like: *) Python socket network programming *) [*args and **kwargs in python explained][] *) Making a url shortener in python","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/the-self-variable-in-python-explained","title":"The self variable in python explained"},{"text":"Hi there fellows. In this post I am going to take you on an adventure with python sockets. They are the real backbones behind web browsing. In simpler terms there is a server and a client. We will deal with the client first. So lets first begin by importing the socket library and making a simple socket. import socket s = socket . socket ( socket . AF_INET , socket . SOCK_STREAM ) Here we made a socket instance and passed it two parameters. The first parameter is AF_INET and the second one is SOCK_STREAM. AF_INET refers to the address family ipv4. ipv6 requires something different but we won't be focusing on it. Secondly the SOCK_STREAM means connection oriented TCP protocol. Now we will connect to a server using this socket. Connecting to a server : Well firstly let me tell you that if any error occurs during the creation of a socket then a socket.error is thrown and secondly we can only connect to a server by knowing it's ip. You can find the ip of the server by doing this in the command prompt or the terminal: $ ping www . google . com You can also find the ip using python: import socket ip = socket . gethostbyname ( 'www.google.com' ) print ip So lets begin with writing our script. In this script we will connect with google. Here's the code for it: import socket # for socket import sys try: s = socket . socket ( socket . AF_INET , socket . SOCK_STREAM ) print \"Socket successfully created\" except socket . error as err: print \"socket creation failed with error %s\" % ( err ) # default port for socket port = 80 try: host_ip = socket . gethostbyname ( 'www.google.com' ) except socket . gaierror: # this means could not resolve the host print \"there was an error resolving the host\" sys . exit () # connecting to the server s . connect (( host_ip , port )) print \" the socket has successfully connected to google on port == %s\" %(host_ip) Now save this script and then run it. You will get something like this in the terminal: Socket successfully created the socket has successfully connected to google on port == 173.194.40.19 So what did we do here ? First of all we made a socket. Then we resolved google's ip and lastly we connected to google. This was not useful for us though so lets move on. So now we need to know how can we send some data through a socket. For sending data the socket library has a sendall function. This function allows you to send data to a server to which the socket is connected and server can also send data to the client using this function. Now lets make a simple server-client program to see all of this in action and hopefully it will make your concepts more clear. Making a server : First of all let me explain a little bit about a server. A server has a bind() method which binds it to a specific ip and port so that it can listen to incoming requests on that ip and port. Next a server has a listen() method which puts the server into listen mode. This allows the server to listen to incoming connections. And lastly a server has an accept() and close() method. The accept method initiates a connection with the client and the close method closes the connection with the client. Now lets begin making our simple server: # first of all import the socket library import socket # next create a socket object s = socket . socket () print \"Socket successfully created\" # reserve a port on your computer in our # case it is 12345 but it can be anything port = 12345 # Next bind to the port # we have not typed any ip in the ip field # instead we have inputted an empty string # this makes the server listen to requests # coming from other computers on the network s . bind (( '' , port )) print \"socket binded to %s\" % ( port ) # put the socket into listening mode s . listen ( 5 ) print \"socket is listening\" # a forever loop until we interrupt it or # an error occurs while True: # Establish connection with client. c , addr = s . accept () print 'Got connection from' , addr # send a thank you message to the client. c . send ( 'Thank you for connecting' ) # Close the connection with the client c . close () So what are we doing here ? First of all we import socket which is necessary. Then we made a socket object and reserved a port on our pc. After that we binded our server to the specified port. Passing an empty string means that the server can listen to incoming connections from other computers as well. If we would have passed 127.0.0.1 then it would have listened to only those calls made within the local computer. After that we put the server into listen mode. What does 5 mean ? It means that 5 connections are kept waiting if the server is busy and if a 6th socket trys to connect then the connection is refused. Lastly we make a while loop and start to accept all incoming connections and close those connections after a thank you message to all connected sockets. Now we have to program a client. Making a client : Now we need something with which a server can interact. We could tenet to the server like this just to know that our server is working. Type these commands in the terminal: # start the server $ python server . py # keep the above terminal open # now open another terminal and type: $ telnet localhost 12345 After typing those commands you will get the following output in your terminal: # in the server.py terminal you will see # this output: Socket successfully created socket binded to 12345 socket is listening Got connection from ( '127.0.0.1' , 52617 ) # In the telnet terminal you will get this: Trying :: 1 ... Trying 127.0.0.1 ... Connected to localhost . Escape character is '^]' . Thank you for connectingConnection closed by foreign host . This output shows that our server is working. Now lets make our client: # Import socket module import socket # Create a socket object s = socket . socket () # Define the port on which you want to connect port = 12345 # connect to the server on local computer s . connect (( '127.0.0.1' , port )) # receive data from the server print s . recv ( 1024 ) # close the connection s . close () The above script is self explanatory but still let me explain to the beginners. First of all we make a socket object. Then we connect to localhost on port 12345 (the port on which our server runs) and lastly we receive data from the server and close the connection. Was that difficult ? I hope not. Now save this file as client.py and run it from the terminal after starting the server script: # start the server: $ python server . py Socket successfully created socket binded to 12345 socket is listening Got connection from ( '127.0.0.1' , 52617 ) $ python client . py Thank you for connecting I hope this intro to sockets in python was digestible for beginners. In my server and client script I have not included exception handling as it would have boggled the beginners. Now I hope that you have a solid understanding about the working of sockets in python. However there's a lot more to it. For further study i recommend the Official python docs and this article on binary tides. If you liked this post then don't forget to share it on facebook, tweet it on twitter and follow our blog. You might also like: *) [*args and **kwargs in python explained][] *) Making a url shortener in python *) 10 inspirations for your next python project","tags":"help, python, tutorial","loc":"http://yasoob.github.io/blog/python-socket-network-programming","title":"Python socket network programming"},{"text":"Hi there folks. I have come to see that most new python programmers have a hard time figuring out the *args and **kwargs magic variables. So what are they ? First of all let me tell you that it is not necessary to write *args or **kwargs. Only the * (aesteric) is necessary. You could have also written *var and **vars. Writing *args and **kwargs is just a convention. So now lets take a look at *args first. Usage of *args *args and **kwargs are mostly used in function definitions. *args and **kwargs allow you to pass a variable number of arguments to a function. What does variable mean here is that you do not know before hand that how many arguments can be passed to your function by the user so in this case you use these two keywords. *args is used to send a non-keyworded variable length argument list to the function. Here's an example to help you get a clear idea: def test_var_args ( f_arg , * argv ): print \"first normal arg:\" , f_arg for arg in argv: print \"another arg through *argv :\" , arg test_var_args ( 'yasoob' , 'python' , 'eggs' , 'test' ) This produces the following result: first normal arg: yasoob another arg through * argv : python another arg through * argv : eggs another arg through * argv : test I hope this cleared away any confusion that you had. So now lets talk about **kwargs Usage of **kwargs **kwargs allows you to pass keyworded variable length of arguments to a function. You should use **kwargs if you want to handle named arguments in a function. Here is an example to get you going with it: def greet_me ( ** kwargs ): if kwargs is not None: for key , value in kwargs . iteritems (): print \"%s == %s\" % ( key , value ) >>> greet_me ( name = \"yasoob\" ) name == yasoob So can you see how we handled a keyworded argument list in our function. This is just the basics of **kwargs and you can see how useful it is. Now lets talk about how you can use *args and **kwargs to call a function with a list or dictionary of arguments. Using *args and **kwargs to call a function So here we will see how to call a function using *args and **kwargs. Just consider that you have this little function: def test_args_kwargs ( arg1 , arg2 , arg3 ): print \"arg1:\" , arg1 print \"arg2:\" , arg2 print \"arg3:\" , arg3 Now you can use *args or **kwargs to pass arguments to this little function. Here's how to do it: # first with *args >>> args = ( \"two\" , 3 , 5 ) >>> test_args_kwargs ( * args ) arg1: two arg2: 3 arg3: 5 # now with **kwargs: >>> kwargs = { \"arg3\" : 3 , \"arg2\" : \"two\" , \"arg1\" : 5 } >>> test_args_kwargs ( ** kwargs ) arg1: 5 arg2: two arg3: 3 Order of using *args **kwargs and formal args So if you want to use all three of these in functions then the order is some_func ( fargs , * args , ** kwargs ) I hope you have understood the usage of *args and **kwargs. If you have got any problems or confusions with this then feel free to comment below. For further study i suggest the official python docs on defining functions and [*args and **kwargs on stackoverflow][]. You might also like : *) Making a url shortener in python *) 20 Python libraries you can't live without *) Targeting python 2 and 3 at the same time.","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/args-and-kwargs-in-python-explained","title":"*args and **kwargs in python explained"},{"text":"Hi there fellas. Today in this post i am going to show you how we can use python to make bulk urls tiny. Let me clear that we are not making a url shortening SERVICE instead what we are going to do is that we are going to unofficially use the tinyurl api (Tinyurl is a url shortening service). There's not any official python api released by tinyurl. So lets begin with this: The First Step : So first of all we have to do some imports. We have to import 7 libraries to make this work. from __future__ import with_statement import contextlib try: from urllib . parse import urlencode except ImportError: from urllib import urlencode try: from urllib . request import urlopen except ImportError: from urllib2 import urlopen import sys We could have imported just one library to make this work but in order to make a good url shortener we have to import these seven libraries. The Second Step : So now we have to start making a method that will handle the url shortening. Watch the code closely because it is self explanatory but however i will explain it later. def make_tiny ( url ): request_url = ( 'http://tinyurl.com/api-create.php?' + urlencode ({ 'url' : url })) with contextlib . closing ( urlopen ( request_url )) as response: return response . read () . decode ( 'utf-8' ) Did you manage to understand the code ? Let me explain it to the beginners. First of all we define a make_tiny function which takes a url as an input. After that we start defining the working of our function. The url_encode takes a url as an input and encodes it i.e escapes it. After that we append the escaped url to the end of tinyurl's api url. Then we open the request_url using urlopen. And lastly we read the response after converting it to utf-8. Why did we convert the response to utf-8 ? The reason for this is that the urlopen function returns a stream of bytes rather than a string so in order to print it and alter it we have to convert it into utf-8. Was that difficult ? So now the next step is to get the input from the user. For this we are going to use the sys library. The Third Step : So lets write the main() function for our little script. Here's the code for it: def main (): for tinyurl in map ( make_tiny , sys . argv [ 1 :]): print ( tinyurl ) So what are we doing here ? We are getting the user input by using sys.argv . We have not limited ourself to only one url as an input instead we are saying that give us as many urls as you want, we will crush them and make them tiny. What sys.argv[1:] does is that it leaves the first two arguments (counting starts from 0) and takes all the rest of the arguments and produces a list of those arguments. For example if you type: $ python script . py url1 url2 url3 then sys.argv[1:] will leave python and script.py and will produce the following list: [ url1 , url2 , url3 ] Wait! what is that map() function over there ? Most beginners will feel confused as most of them have never used map. Map() is a simple way of looping over a list and passing it to a function one by one. The map() function above there is equivalent to : def main (): urls = sys . argv [ 1 :] for url in urls: tinyurl = make_tiny ( url ) print tinyurl I hope the above code example cleared away any confusions about map() function. The Fourth Step : Now lets wrap up our code. The only thing left is this : if __name__ == '__main__' : main () Add this to the end of your code. This tells us whether the script was executed independently from the shell or was it called by another script. This is pretty handy if you want to use this script later in any other project. Finally : So here's the complete script: from __future__ import with_statement import contextlib try: from urllib . parse import urlencode except ImportError: from urllib import urlencode try: from urllib . request import urlopen except ImportError: from urllib2 import urlopen import sys def make_tiny ( url ): request_url = ( 'http://tinyurl.com/api-create.php?' + urlencode ({ 'url' : url })) with contextlib . closing ( urlopen ( request_url )) as response: return response . read () . decode ( 'utf-8' ) def main (): for tinyurl in map ( make_tiny , sys . argv [ 1 :]): print ( tinyurl ) if __name__ == '__main__' : main () Considering that you have saved this script as url_shortener.py you have to run it like this from the shell: $ python url_shortener . py url1 url2 url3 If you want to save these tinyurls into a txt file then issue this command: $ python url_shortener . py url1 > urls . txt I hope you liked today's post. This script can work on python 2 and python 3 both . It was primarily aimed at two kind of audience. Firstly those who are learning python and want to learn how to make a simple yet useful script and those who want to learn how to make a url shortener in python. Do share your view in the comments bellow and stay tuned for the next post. If you want regular dose of python tips and tutorials delivered to your doorstep then consider following my blog. You might also like: * 5 quality screencasts worth watching * Packaging and distributing your python libraries * 10 inspirations for your next python project","tags":"python, tutorial","loc":"http://yasoob.github.io/blog/a-url-shortener-in-python","title":"Making a url shortener in python"},{"text":"Hello there fellas. I am back with another list of useful resources. This time i am going to share with you 5 quality python screen casts worth watching. Unlike ruby there are not really much quality screen casts related to python out there. So without wasting a dime lets continue with the list of those screen casts. So here it is: 1. The first one if Encoding Specifiers and Magical Mode Lines in python. There's a lot in it for learning. Go check it out and you won't regret it. 2. The second one is by Neck Beard Republic. This one is about lambda functions in python . Neck Beard Republic have great python screen casts but most of them are paid however they have some free screen casts as well. 3. The third one is a series of screen casts by Kenneth Love. They are related to django. The title is Getting Started With Django. They are great. I have personally watched them. There's something for everyone in it. 4. The fourth one is again by Neckbeard Republic. This one is a must watch for every python programmer out there. The title of this screen cast is [__dunder__ functions in python][]. There can be alot of mystery involved in Python when it comes to the dunder (double underscore) methods. This screen cast will reveal the secret behind these function. However i have not personally watched it but it's on my todo list. 5. Ah so now comes the last one. This one is again a Neckbeard Republic production. This one is about the super() literal in python . I can see how much effort Mahdi Yusuf has put into making these screen casts. So now we come to the end of another great post about 5 quality python screen casts worth watching. However there are some other video resources as well like Showmedo which i will cover in a future post. If you are feeling adventurous then check out 10 inspirations for your next python project or Packaging and distributing your python libraries . I hope you liked today's post. Do share your views in the comments below and stay tuned for the next post.","tags":"inspiration, python","loc":"http://yasoob.github.io/blog/5-quality-screencasts-worth-watching","title":"5 quality screencasts worth watching"},{"text":"Hi there fellas. In this post i am going to tell you about pickle. It is used for serializing and de-serializing a Python object structure. Any object in python can be pickled so that it can be saved on disk. What pickle does is that it \"serialises\" the object first before writing it to file. Pickling is a way to convert a python object (list, dict, etc.) into a character stream. The idea is that this character stream contains all the information necessary to reconstruct the object in another python script. So lets continue: 1. First of all you have to import it through this command: import pickle pickle has two main methods. The first one is dump, which dumps an object to a file object and the second one is load, which loads an object from a file object. 2. Prepare something to pickle: Now you just have to get some data which you can pickle. For the sake of simplicity i will be pickling a python dictionary. So just read the below code and you will be able to figure it out yourself. import pickle a = [ 'test value' , 'test value 2' , 'test value 3' ] a [ 'test value' , 'test value 2' , 'test value 3' ] file_Name = \"testfile\" # open the file for writing fileObject = open ( file_Name , 'wb' ) # this writes the object a to the # file named 'testfile' pickle . dump ( a , fileObject ) # here we close the fileObject fileObject . close () # we open the file for reading fileObject = open ( file_Name , 'r' ) # load the object from the file into var b b = pickle . load ( fileObject ) b [ 'test value' , 'test value 2' , 'test value 3' ] a == b True The above code is self explanatory. It shows you how to import the pickled object and assign it to a variable. So now we need to know where we should use pickling. It is very useful when you want to dump some object while coding in the python shell. So after dumping whenever you restart the python shell you can import the pickled object and deserialize it. But there are other use cases as well which i found on stackoverflow . Let me list them below. 1) saving a program's state data to disk so that it can carry on where it left off when restarted (persistence) \\2) sending python data over a TCP connection in a multi-core or distributed system (marshalling) \\3) storing python objects in a database \\4) converting an arbitrary python object to a string so that it can be used as a dictionary key (e.g. for caching & memoization). One thing to note is that there is a brother of pickle as well with the name of cpickle. As the name suggests it is written in c which makes it 1000 times more faster than pickle. So why should we ever use pickle instead of cpickle ? Here's the reason >>> Because pickle handles unicode objects. >>> Because pickle is written in pure Python, it's easier to debug. For further reading i suggest the official pickle documentation or if you want to read more tutorials then check out the sqlite tutorial . Now we come to the end of today's post. I hope you liked it. Do follow my blog to get regular updates. If you have any suggestions or comments then post them below.","tags":"python","loc":"http://yasoob.github.io/blog/what-is-pickle-in-python","title":"What is Pickle in python ?"},{"text":"Hi there fellow coders. I know those times when we don't know what to code. I am sure you must have experienced that time as well. Today i am going to share with you 10 python projects that will serve as an inspiration for your next python project. So without wasting a time here's the list. 1. Pillow . Pillow is the \"friendly\" PIL fork by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lundh and Contributors. Alex teaches us that if you don't like something then make a fork. 2. Inbox.py . This is the simplest SMTP server you'll ever see. It's asynchronous. One instance should handle over one thousand emails per second. Pretty impressive. 3. PyMinecraft . Have you ever thought of implementing the famous minecraft game into python ? Hawstein has. 4. fuqit . The latest and greatest from Zed Shaw (the writer of learn python the hard way) , fuqit is a web framework that attempts to make you forget everything you have learned from MVC frameworks and focuses on simplicity. Zed Shaw teaches us one thing through this project \"Don't panic while doing something against the trend\". 5. PyPI Notifier . Watches your requirements.txt files and notifies you via email when a project is updated. Although i have not used it myself but i think this project is great. 6. Vol . A simple project but great inspiration. vol lets you control your mac's audio volume from the commandline. It also has support for profiles. 7. Youtube-dl . A great python project. It allows you to download videos from various websites. It's a great project make sure that you check it out. 8. Jukebox . Ever wanted to listen to music with a larger group of people e.g. in your office? Who decides what to play? Make your music player democratic and give everyone the chance to promote their favourite song. This script will relieve you from numerous fights. 9. Mitmproxy . mitmproxy is a an SSL capable man-in-the-middle-proxy. It allows HTTP(and other) traffic to be inspected and edited on the fly. Don't be caught unaware by apps like Path any more, now you can know what data you are sending and receiving from remote servers. 10. Thumbor . It is a smart imaging service. It enables on-demand crop, resizing and flipping of images. I haven't checked it out though but it seems promising. Do check it out if you want a server side image cropper. So finally we come to the end of this post. I hope you found some useful python projects in there to serve as an inspiration. Share your views in the comments below and tell me which project you liked the most or if you want to read further then check out my another blog post about 20 python libraries you can't live without .","tags":"inspiration, python","loc":"http://yasoob.github.io/blog/10-inspirations-for-your-next-python-project","title":"10 inspirations for your next python project"},{"text":"Hi there fellow coders. This post will go over the basics of packaging and deploying your python libraries. So without wasting a minute lets get started . Folder structure So first thing first. What should be your folder structure ? At minimum you can have only two files. The first one should be the setup.py file and the second one should be your module file. But today I am not going to talk about the very basics. Today we are going to follow this folder structure : My_module / LICENSE . txt README . txt setup . py My_module / __init__ . py Here I have 4 files. The License file contains the license under which you want to distribute your module. So lets move on. What should be our second step ? Obviously it should be to describe our module and write some meta data into the setup.py file. Describing your module The setup.py file is the heart of any python module or library. It describes the module and lists some other useful info about the module like it lists any dependencies on which a module depends and it also tells distutils where to find the necessary scripts of this module. So lets describe our module with the help of our setup.py file. from distutils . core import setup setup ( name = 'My super module' , version = '0.1dev' , packages = [ 'My_module' ,], license = 'Creative Commons Attribution-Noncommercial-Share Alike license' , long_description = open ( 'README.txt' ) . read (), ) Here we could have rewritten the long_description ourselves but we reused our README file. We wrote 'dev' in the version because we still do not have anything in our module but we are moving toward the 0.1 release. When you have enough code in your module then feel free to drop 'dev' from the version field. Now the next step is to make our first release. Making your first release So how do we make our first release ? Just follow me. Your release should have a single archive file. It can be easily made with this command. $ python setup . py sdist Just go to the root of your module folder and execute this command. It will create a subdirectory with the name of dist and will package all of your module scripts and other files into a single archive file ready to be uploaded on PyPI (Python Package Index). This archive file will have the default extension of zip on windows and tar.gz on POSIX systems. Publishing your module Now that archive file can be uploaded anywhere for distribution but we will focus on PyPI. In order to upload on PyPI you will first have to make an account on http://pypi.python.org/pypi . After that you will have to register your package on PyPI like this: $ cd path /to/ My_module $ python setup . py register Just use your existing login. Afte that run the following command to upload it to PyPI. $ python setup . py sdist upload This will make the distribution one time more and will upload it to PyPI. Now you should be proud of yourself because you have just contributed to the opensource world. I hope you liked my post. If you want to read about setup.py then go here . For further study i recommend the Hichhikers guide to packaging . I hope you liked today's post. Do share your views in the comments below and stay tuned for the next post.","tags":"installation, packaging, python","loc":"http://yasoob.github.io/blog/packaging-and-distributing-your-python-libraries","title":"Packaging and distributing your python libraries"},{"text":"Hi there fellas. Today i am going to teach you how to use sqlite databases with python. This post will cover the basics of making and using a sqlite database with python using the sqlite3 library. Okay lets get started. Firstly if you are using python 2.5 or greater then you will have sqlite3 installed otherwise you will have to install it. Creating and connecting to a database So how do you make a database in python using the sqlite3 library ? It's pretty simple. Just follow the code below and you will be able to make it out on your own. 1 2 3 4 5 6 7 8 9 #!/usr/bin/python import sqlite3 # If the database is already created then this # code will connect to it instead of making a new one conn = sqlite3 . connect ( 'test.db' ) print \"Created database successfully\" So was that difficult ? I hope not. So lets continue. The next thing is to make tables in our database. So how do you go about doing it ? Just follow me. Making tables in a database import sqlite3 conn = sqlite3 . connect ( \"test.db\" ) with conn: cursor = conn . cursor () # create a table cursor . execute ( \"\"\"CREATE TABLE books (title text, author text)\"\"\" ) In the above code we made a table with the name of book. It has the following fields : title and author. Both of these fields have the data type of text. First of all we made a database with the name of test.db and after that we made a cursor object which allows us to interface with our database and execute queries. So what now. We have created a database and made a table. Now we have to insert some data in our table. Lets continue. Inserting data to the database # insert some data cursor . execute ( \"INSERT INTO books VALUES ('Pride and Prejudice', 'Jane Austen')\" ) # save data to database conn . commit () # insert multiple records using the more secure \"?\" method books = [( 'Harry Potter' , 'J.K Rowling' ), ( 'The Lord of the Rings' , 'J. R. R. Tolkien' ), ( 'The Hobbit' , 'J. R. R. Tolkien' )] cursor . executemany ( \"INSERT INTO books VALUES (?,?)\" , books ) conn . commit () So in the above code i showed you two ways to put some data into the database. The first method is to execute a single query and the second method is to execute multiple queries in the same time. In the second method we could have used the string substitution '%s' but it is known to be potentially dangerous and can lead to SQL Injection. So whats left now ? Removing and updating the data ? No problem i will cover that as well. Just examine the code below. Updating data in the database import sqlite3 conn = sqlite3 . connect ( \"test.db\" ) with conn: cursor = conn . cursor () sql = \"\"\" UPDATE books SET author = 'Yasoob' WHERE author = 'J.K Rowling' \"\"\" cursor . execute ( sql ) In the above code we updated our record by replacing J.K Rowling with Yasoob. Take a look at the below code for deleting the records. Deleting records from the database import sqlite3 conn = sqlite3 . connect ( \"test.db\" ) with conn: cursor = conn . cursor () sql = \"\"\" DELETE FROM books WHERE author = 'Yasoob' \"\"\" cursor . execute ( sql ) In the above code we deleted the record of those books whose writer was 'Yasoob'. Now i am going to show you how to display data from the table. It is easy. Just a few lines of code. Displaying data from the database import sqlite3 conn = sqlite3 . connect ( 'test.db' ) print \"Opened database successfully\" ; cursor = conn . execute ( \"SELECT title, author from books\" ) for row in cursor: print \"Title = \" , row [ 0 ] print \"Author = \" , row [ 1 ], \"\\n\" print \"Operation done successfully\" ; conn . close () In the above code we opened our database file and displayed the records on the screen. Do you know that we could have also done: conn = sqlite3 . connect ( ':memory:' ) So what is this code doing ? The :memory: is a special name which creates the database in the ram and lets you execute any query. Lastly I would like to tell you about sqlalchemy which is a database library for python and takes care of a lot of things for you. It does all the escaping for you so you won't have to mess with the annoyances of converting embedded single quotes into something that SQLite will accept. It's a part of my previous blog post as well. So now goodbye to you all. I hope you liked reading today's post as much as i enjoyed writing it. Stay tuned for my next post. For further reading i suggest the zetcode database tutorial .","tags":"python, server","loc":"http://yasoob.github.io/blog/connecting-to-sqlite-databases","title":"Connecting to Sqlite databases"},{"text":"Hi there fellas. Today i am going to ask you for a favour. I am going to write an ebook about python. I want you to give me some ideas about what i should write. This Ebook is going to be short. It will be between 10 to 20 pages because it's my first ebook. Currently i have two topics in mind. The first one is to write an introductory ebook about how to get started with python. This will be for beginners and will include the installation and setting up of python as well as some introductory tutorials. The other topic is to write about how i made the soundcloud searcher and downloader . All the backend programming of soundcloud searcher and downloader is done in python. This ebook will contain all the methods and resources I used to make this soundcloud searcher and downloader as well as how I got it working on google app engine. In my opinion the second topic will be more useful for you guys as most of us are looking for these type of ebooks. What do you say ? If you want me to write about any other topic then do tell me in the comments below and I will see.","tags":"help, python","loc":"http://yasoob.github.io/blog/give-me-an-idea-for-an-ebook","title":"Give me an idea for an Ebook"},{"text":"Hi there fellow programmers. I remember those days when i started learning python. I was always eager to find new blogs about python but found just a handful of them. This is one of the reason i started this blog. To spread my knowledge about python to other fellow coders. Today i am going to try to provide you 10 links to different python blogs that are worth following. By that i mean they are regularly updated and have useful info. So here's the list: 1. The first one is Planet python undoubtedly one of the most famous python blogs out there. Go check it out. I hope you will find it helpful. It was last updated today. 2. The second blog is lucumr . The home of the developer of flask (a python micro web framework). 3. The third blog is love-python . I don't know how i stumbled over it but i really like this blog as it has a lot of useful info and code snippets. 4. The fourth blog is of Doug Hellmann . He is the one behind PYMOTW (Python Module Of The Week). This blog covers a lot of python libraries. If you are searching for some new useful python libraries then go check this out. 5. The next blog is Coder Who Says Py . It's a nice blog. Although it's not regularly updated but still you will find some helpful info in there. 6. The sixth blog is effbot . Full of helpful info, code snippets and everything else you can expect from a python blog. Go check it out before it's too late. 7. The seventh blog is of the writer of \"Two Scoops Of Django\". It is pydanny . This is mainly about django but still a lot can be learned from this blog. 8. The next blog is of Al Sweigat. He is the writer of numerous python books including his latest book \"Hacking Secret Ciphers With Python\". 9. The ninth blog is this blog on which you currently are. This is your all in one guide to python. 10. The last blog is one of the most useful blog which got me up to speed with python. It is called Mouse Vs Python . I hope you liked today's post and i am sure i helped you out of your frustration of finding good python blogs. Do share your views in the comments below and if you think that i have missed a blog then put it in the comments as well.","tags":"python, welcome","loc":"http://yasoob.github.io/blog/10-python-blogs-worth-following","title":"10 python blogs worth following"},{"text":"Hi there fellas. Today i am going to list 20 python libraries which have been a part of my toolbelt and should be a part of yours as well. So here they are : 1. Requests . The most famous http library written by kenneth reitz. It's a must have for every python developer. 2. Scrapy . If you are involved in webscraping then this is a must have library for you. After using this library you won't use any other. 3. wxPython . A gui toolkit for python. I have primarily used it in place of tkinter. You will really love it. 4. Pillow . A friendly fork of PIL (Python Imaging Library). It is more user friendly than PIL and is a must have for anyone who works with images. 5. SQLAlchemy . A database library. Many love it and many hate it. The choice is yours. 6. BeautifulSoup . I know it's slow but this xml and html parsing library is very useful for beginners. 7. Twisted . The most important tool for any network application developer. It has a very beautiful api and is used by a lot of famous python developers. 8. NumPy . How can we leave this very important library ? It provides some advance math functionalities to python. 9. SciPy . When we talk about NumPy then we have to talk about scipy. It is a library of algorithms and mathematical tools for python and has caused many scientists to switch from ruby to python. 10. matplotlib . A numerical plotting library. It is very useful for any data scientist or any data analyzer. 11. Pygame . Which developer does not like to play games and develop them ? This library will help you achieve your goal of 2d game development. 12. Pyglet . A 3d animation and game creation engine. This is the engine in which the famous python port of minecraft was made 13. pyQT . A GUI toolkit for python. It is my second choice after wxpython for developing GUI's for my python scripts. 14. pyGtk . Another python GUI library. It is the same library in which the famous Bittorrent client is created. 15. Scapy . A packet sniffer and analyzer for python made in python. 16. pywin32 . A python library which provides some useful methods and classes for interacting with windows. 17. nltk . Natural Language Toolkit \u2013 I realize most people won't be using this one, but it's generic enough. It is a very useful library if you want to manipulate strings. But it's capacity is beyond that. Do check it out. 18. nose . A testing framework for python. It is used by millions of python developers. It is a must have if you do test driven development. 19. SymPy . SymPy can do algebraic evaluation, differentiation, expansion, complex numbers, etc. It is contained in a pure Python distribution. 20. IPython . I just can't stress enough how useful this tool is. It is a python prompt on steroids. It has completion, history, shell capabilities, and a lot more. Make sure that you take a look at it. I hope you liked today's post. Do share your views in the comments below and if you think that i have missed an important library which should be listed up there post it in the comments and i will add it. Stay tuned for the next post.","tags":"installation, python","loc":"http://yasoob.github.io/blog/20-python-libraries-you-cant-live-without","title":"20 Python libraries you can't live without"},{"text":"Hi there pythonistas. Today i am not going to write a lengthy tutorial about how to make your programs compatible with python 2 and 3 at the same time, but i am going to share a simple tip with you guys which will help you achieve this target. Just imagine that you have a very popular python module which is use by hundreds of people but not all of them have python 2 or 3. In that case you have two choices. The first one is to distribute 2 modules. One for python 2 and the other for python 3 and the other choice is to modify your current code so that it can be used with both python 2 and 3. I am going to talk about the second choice. First tell me how you import packages in your script ? Most of us do this : import foo # or from foo import bar Do you know that you can do something like this as well? import foo as foo I know it's function is the same as above listed code but it is vital for making your script compatible with python 2 and 3. Now examine the code below : try : import urllib.request as urllib_request #for python 3 except ImportError : import urllib2 as urllib_request # for python 2 Did you find something noteworthy in the above code ? So let me explain the above code a little. We are wrapping our importing code in a try except clause. Why are we doing that ? We are doing it because in python2 there is no urllib.request module and will result in an ImportError. The functionality of urllib.request is provided by urllib2 module in python2. So now when python2 try to import urllib.request and get an import error we tell it to import urllib2 instead. The final thing you need to know about is the as keyword. It is mapping the imported module to urllib_request. So that now all of the Classes and methods of urllib2 are available to us by urllib_request. So how did this help us in targeting python 2 and 3 at the same time ? Okay here is the trick. Once you have got a library that works in python 2 but is not available in python 3 then you have to find out it's alternative in python 3 (most libraries have their alternatives for python 3 but some don't). I know it's trivial and that is why a lot of modules have still not been ported to python 3. After that you have to wrap your module importing code in a try-except clause with a common variable following the as keyword. After that use the imported module. Here is a complete example which should work both in python 2 and 3: try : import urllib.request as compat_urllib_request except ImportError : # Python 2 import urllib2 as compat_urllib_request html = compat_urllib_request . urlopen ( \"http://www.google.com/\" ) print html . headers [ 'content-type' ] 'text/html; charset=ISO-8859-1' Another example which involves the json library: try : import json except ImportError : import simplejson as json I hope you liked today's post. Stay tuned for the next one. If you have any suggestions then do post them in the comments below.","tags":"python","loc":"http://yasoob.github.io/blog/make-your-programs-compatible-with-python-2-and-3-at-the-same-time","title":"Targeting python 2 and 3 at the same time."},{"text":"Okay so the chances are you already know about exception handling in python but some new programmers don't. How do you handle exceptions in python ? First let me tell you what exceptions are. Exceptions are when something unexpected happens with your code. Just think that you writing a huge program which browses the internet and accesses web pages, the program works fine on your side and you consider packaging and distributing it. But something unexpected occurs on the pc of the user who downloaded your package from the internet and tried to run it. His internet connection dropped. What will happen ? >>> import urllib2 >>> urllib2 . urlopen ( 'http://www.google.com/' ) . read () Traceback ( most recent call last ): File \"\" , line 1 , in urllib2 . urlopen ( 'http://www.google.com/' ) . read () File \"C:\\Python27\\lib\\urllib2.py\" , line 127 , in urlopen return _opener . open ( url , data , timeout ) File \"C:\\Python27\\lib\\urllib2.py\" , line 404 , in open response = self . _open ( req , data ) File \"C:\\Python27\\lib\\urllib2.py\" , line 422 , in _open '_open' , req ) File \"C:\\Python27\\lib\\urllib2.py\" , line 382 , in _call_chain result = func ( * args ) File \"C:\\Python27\\lib\\urllib2.py\" , line 1214 , in http_open return self . do_open ( httplib . HTTPConnection , req ) File \"C:\\Python27\\lib\\urllib2.py\" , line 1184 , in do_open raise URLError ( err ) So in order to save yourself from this you have to use Exception Handling . There are three keywords available within python which help you to handle exceptions. They are usually referred to as try-except clause. Just watch the below code. >>> try: urllib2 . urlopen ( 'http://www.google.com/' ) . read () except urllib2 . URLError as e: print \"An error \" An error >>> Here we are doing the same thing but using the try except clause. Lets get somewhat deeper into it. We wrap our main code in the try clause. After that we wrap some code in except clause which gets executed if an exception occurs in the code wrapped in try clause. But wait where is the third keyword ? The third keyword is finally . It comes after the except clause and gets executed even if no exception occurs. Let me show you the same example but this time with the finally clause as well. >>> try: urllib2 . urlopen ( 'http://www.google.com/' ) . read () except urllib2 . URLError as e: print \"An error %s \" %e finally: print \"This is going to be printed even if no exception occurs\" ' Google ( function (){ \\ nwindow . google = { kEI: \"sdD2UbmEOMzBtAbyqYCgDQ\" , ge ------------------------ trancuated ------------------- This is going to be printed even if no exception occurs So as you can see the finally clause got executed even when there was no exception. Okay now i am going to tell you a little secret which most of us don't know and even i got to know it just few days ago. While using the except clause if we want to handle any exception that we are not aware of then what should we do ? Here in this situation we can put more except clauses for each exception which is likely to occur and in the end we can put another except clause which caches all exceptions which were not likely to occur. Here is an example : >>> try: print ali except KeyboardInterrupt: print \"'E's pining!\" except : print \"This is going to handle every type of exception\" finally: print \"okay the end\" This is going to handle every type of exception okay the end Okay i guess thats it for now. However i just gave you an intro of exception handling in python and if you want to study further then go to this link. I hope you liked today's post. Do share your views about todays post in the comments below. Source : http://docs.python.org/2/tutorial/errors.html","tags":"python","loc":"http://yasoob.github.io/blog/exception-handling","title":"Exception handling"},{"text":"Have you ever wondered how to check the install path of python and it's version ? If you have then this post is for you. It is very easy to check the version and install location of python on linux as well as on windows. First I will show the method of checking this on linux and then on windows. 1. On linux just type these two commands in the terminal: # for python install path root @bt: / $ which python /usr/ bin / python # for python version; root @bt: /usr/ bin $ python - V Python 2.7.5 2. On windows type this commands in the command prompt for python version: python - V And type these commands in the python shell for python install path >>> import os >>> import sys >>> os . path . dirname ( sys . executable ) 'C:\\\\Python27' This comes in handy when you need to know the version and path of python real quick. I hope you liked today's post. Stay tuned for the next one. Do share your views in the comments below.","tags":"installation, python","loc":"http://yasoob.github.io/blog/python-install-location-and-version","title":"Python install location and version"},{"text":"Yet another useful post and this one is about coding styles in python. Everyone has his own coding style in python. Here i am going to post some links that will help you to choose your own coding style in python. The first link is undoubtedly from the official python documentation as python documentation has everything to get you started. This link will teach you how to use python and how not to use python. http://docs.python.org/2/howto/doanddont.html The second link is from the famous PEP-8 which is python's official style guide. It determines the python's official coding standards. You should definitely take a look at them and try to incorporate them in your code. http://www.python.org/dev/peps/pep-0008/ The third link is from google because as you know google is a huge user of python and most of their infrastructure is based on python as is their online support forum. So it's not surprising that google has it's own style guide for python. http://google-styleguide.googlecode.com/svn/trunk/pyguide.html So the last link is from khanacademy . As you know it's the biggest online academy and whole of their online infrastructure is based on python. I really love their python style guide. https://sites.google.com/a/khanacademy.org/forge/for-developers/styleguide/python I hope you liked this post. Do share your views in the comments below.","tags":"python","loc":"http://yasoob.github.io/blog/coding-styles-for-python","title":"Coding styles for python"},{"text":"Hi there pythonistas. Most of the new python programmers don't know how to correctly install python on windows. Today I am going to share clean and clear instructions on how to install python on a Windows 7 machine. There are only 4 steps involved so without wasting a minute lets get started. 1. Visit the official Python download page and grab the Windows installer. Choose the 32-bit version. (A 64-bit version is available, but there are compatibility issues with some modules you may want to install later.) Note: Python currently exists in two versions. One is 2.xx version and the other is 3.xx version. In this post i am going to focus only on 2.xx version as it is more widely used right now and some modules don't support 3.xx version as yet. 2. Run the installer and accept all the default settings. Don't change the default directory it creates. 3. Now comes the trickiest part. We have to add python to our environment variable. You do not have to understand what environment variable is. Just follow along with me. We set the system's PATH variable to include directories that include Python components and packages so that they are available in the command prompt. Without doing this we won't be able to use python from command line. To do this: Right-click Computer and select Properties. In the dialog box, select Advanced \u00c2 System Settings. In the next dialog, select Environment Variables. In the User Variables section, edit the PATH statement to include this: C: \\ Python27 ; C: \\ Python27 \\ Lib \\ site - packages \\ ; C: \\ Python27 \\ Scripts \\ ; 4. Now, you can open a command prompt (Start Menu|Accessories or Start Menu|Run|cmd) and type: C: \\> python That will load the Python interpreter: Python 2.7.3 ( default , Apr 10 2012 , 14 : 24 ) [ MSC v .1500 32 bit ( Intel )] on win32 Type \"help\" , \"copyright\" , \"credits\" or license for more information . >>> Just because of the changes that you did to the path variable you are now able to use python from command prompt. Now press Control-Z to exit the interpreter and get back to a C: prompt. Till now the the installation of python is complete but you may want to install setuptools which offers the helpful easy_install utility for installing Python packages. Grab the appropriate version for your system and install and that's it for now. Stay tuned for the next post.","tags":"installation, python","loc":"http://yasoob.github.io/blog/setting-up-python-on-windows","title":"Setting up python on windows"},{"text":"Oh hi there! Welcome to another useful post. This post is going to be about how to package your python scripts and packages for distribution on PyPI or some other place. Here I won't go too deep into explaining everything as most of us just need to know the basics of packaging. However i will provide you with different links for further study. Okay lets talk about >>> setuptools first. What is it ? It's a Python module which allows us to easily package our python scripts and modules for distribution. However there are other packaging libraries as well but here i will talk about setuptools only. So what should be the basic example to show the usage of setuptools ? Here you go. For basic use of setuptools, just import things from setuptools and then look below for the minimal setup script using setuptools. from setuptools import setup , find_packages setup ( name = \"HelloWorld\" , version = \"0.1\" , packages = find_packages (), ) As you see we don't have to specify much in order to use setuptools in a project. Just by doing the above, this project will be able to produce eggs, upload to PyPI, and automatically include all packages in the directory where the setup.py lives. But when you are releasing your projects on PyPI then you should add a bit more information about yourself and this package and if your project relies on some external dependencies then list them there as well. Here is another script which can do all that : from setuptools import setup , find_packages setup ( name = \"HelloWorld\" , version = \"0.1\" , packages = find_packages (), scripts = [ 'say_hello.py' ], # Project uses reStructuredText, so ensure that the # docutils get installed or upgraded on the target # machine install_requires = [ 'docutils>=0.3' ], package_data = { # If any package contains *.txt or *.rst files, # include them: '' : [ '*.txt' , '*.rst' ], # And include any *.msg files found in the # 'hello' package, too: 'hello' : [ '*.msg' ], }, # metadata for upload to PyPI author = \"Me\" , author_email = \"me@example.com\" , description = \"This is an Example Package\" , license = \"PSF\" , keywords = \"hello world example examples\" , # project home page, if any : url = \"http://example.com/HelloWorld/\" , # could also include long_description, download_url, # classifiers, etc. ) I hope this is enough for now. However here are some other packaging libraries in case you were wondering : Distutils is the standard tool used for packaging. It works rather well for simple needs, but is limited and not trivial to extend. Setuptools is a project born from the desire to fill missing distutils functionality and explore new directions. In some sub-communities, it's a de facto standard. It uses monkey-patching and magic that is frowned upon by Python core developers. Distribute is a fork of Setuptools that was started by developers feeling that its development pace was too slow and that it was not possible to evolve it. Its development was considerably slowed when distutils2 was started by the same group. Distutils2 is a new distutils library, started as a fork of the distutils codebase, with good ideas taken from setup tools (of which some were thoroughly discussed in PEPs), and a basic installer inspired by pip. Distutils2 did not make the Python 3.3 release, and it was put on hold. >>> source For further study i recommend: http://pythonhosted.org/distribute/setuptools.html","tags":"python","loc":"http://yasoob.github.io/blog/packaging-your-python-scripts","title":"Packaging your python scripts."},{"text":"Okay here we are with yet another post. However this post is going to be short as in this post i am going to teach you a small, simple and very useful trick. Have you ever tried to access a value from a dictionary and got an exception that there is no value mapped to that key ? Like this : >>> dict = { 'key1' : 2 , 'key2' : 3 } >>> dict [ 'key1' ] 2 >>> dict [ 'key3' ] Traceback ( most recent call last ): File \"\" , line 1 , in dict [ 'key3' ] KeyError: 'key3' Sometimes you do not want anything like this because this error can break your whole program and make it useless until you fix this issue. Is there any way through which we can see if a key is present in a dictionary or not and if it is not present then get a default value in it's place ? the chances are that you don't know of any such method. Okay so here's the trick. Just examine the code below and i will explain it later. >>> dict = { 'key1' : 2 , 'key2' : 3 } >>> dict [ 'key1' ] 2 >>> a = dict . get ( 'key3' , 0 ) >>> print a 0 So what is happening above ? We are still accessing the same key but where has the exception gone ? Let me introduce you to get() method which is available with a dictionary. What this method does is that it allows you to specify a default value which is returned in case the key is not present in the dictionary. In our case there is no key3 in our dict but when we are using the get() method we are specifying the default value which should be returned in case if key3 is not present and the default value which i have set is 0. The syntax of the get method is : dict.get('anykey','the default value to return if the key is not present') . However if we do not set a default value then this returns `None` if the key is not present. I hope it's enough for you guys however if you want to further explore this get() method then google is your friend. Do share your views in the comments below.","tags":"python","loc":"http://yasoob.github.io/blog/dictionaries-have-a-get-method","title":"Dictionaries have a get() method"},{"text":"Okay so here we have another post. This post is about the return keyword. you might have encountered some functions written in python which have a return keyword in the end of the function. Do you know what it does ? Lets examine this little function: >>> def add ( value1 , value2 ): ... return value1 + value2 >>> result = add ( 3 , 5 ) >>> print result 8 The function above takes two values as input and then output their addition. We could have also done: >>> def add ( value1 , value2 ): ... global result ... result = value1 + value2 >>> add ( 3 , 5 ) >>> print result 8 So first lets talk about the first bit of code which involves the return keyword. What that function is doing is that it is assigning the value to the variable which is calling that function which in our case is result . It's pretty handy in most cases and you won't need to use the global keyword. However lets examine the other bit of code as well which includes the global keyword. So what that function is doing is that it is making a global variable result. What does global mean here ? Global variable means that we can access that variable outside the function as well. Let me demonstrate it with an example : # first without the global variable >>> def add ( value1 , value2 ): result = value1 + value2 >>> add ( 2 , 4 ) >>> result # Oh crap we encountered an exception. Why is it so ? # the python interpreter is telling us that we do not # have any variable with the name of result. It is so # because the result variable is only accessible inside # the function in which it is created if it is not global. Traceback ( most recent call last ): File \"\" , line 1 , in result NameError: name 'result' is not defined # Now lets run the same code but after making the result # variable global >>> def add ( value1 , value2 ): global result result = value1 + value2 >>> add ( 2 , 4 ) >>> result 6 So hopefully there were no errors in the second run as expected. If you want to further explore them you should check out the following links. 1. 2. Stackoverflow","tags":"python","loc":"http://yasoob.github.io/blog/the-use-of-return-and-global-keywords","title":"The use of return and global keywords."},{"text":"Okay so most of us do not know how to generate random strings which include letters and digits. This can be really useful for generating a password or something else just use your evil mind. Okay so how do we generate a random string. Have you ever heard of the string module available in python ? The chances are that you have not heard about it. So what does this module provide us ? Okay here you go lets understand this module by example: # first we have to import random module as this # provides the backbone for our random string # generator and then we have to import the string # module. >>> import random >>> import string # now lets see what this string module provide us. # I wont be going into depth because the python # documentation provides ample information. # so lets generate a random string with 32 characters. >>> random = '' . join ([ random . choice ( string . ascii_letters + string . digits ) for n in xrange ( 32 )]) >>> print random '9fAgMmc9hl6VXIsFu3ddb5MJ2U86qEad' >>> print len ( random ) 32 Another example with a function: >>> import string >>> import random >>> def random_generator ( size = 6 , chars = string . ascii_uppercase + string . digits ): ... return '' . join ( random . choice ( chars ) for x in range ( size )) ... >>> random_generator () 'G5G74W' >>> random_generator ( 3 , \"6793YUIO\" ) 'Y3U' So thats it for now. If you want to further explore the string module then go to the official documentation","tags":"python","loc":"http://yasoob.github.io/blog/generating-a-random-string","title":"Generating a random string."},{"text":"Okay yet another useful post. This post is really important and useful for anyone just starting out with python. So what is the __init__.py file ? Files name __init__.py are used to mark directories on disk as Python package directories. If you have the files mydir /spam/ __init__ . py midir /spam/mo dule . py and mydir is on your path, you can import the code in module.py as import spam . module or from spam import module If you remove the __init__.py file, Python will no longer look for submodules inside that directory, so attempts to import the module will fail. The __init__.py file is usually empty, but can be used to export selected portions of the package under more convenient name, hold convenience functions, etc. Given the example above, the contents of the init module can be accessed as import spam And finally here is what the official documentation has to say about this file: The __init__ . py files are required to make Python treat the directories as containing packages ; this is done to prevent directories with a common name , such as string , from unintentionally hiding valid modules that occur later on the module search path . In the simplest case , __init__ . py can just be an empty file , but it can also execute initialization code for the package or set the __all__ variable , described later . Source : http://effbot.org/pyfaq/what-is-init-py-used-for.htm","tags":"python, __init__.py","loc":"http://yasoob.github.io/blog/what-is-__init__-py","title":"What is __init__.py ?"},{"text":"So the chances are that you already know about the with statement but some of us do not know about it. Lets face the reality the with statement saves us a lot of time and reduces our code base. Just imagine you are opening a file width python and then saving something to it. You would normally do : file = open ( 'file.txt' , 'w' ) file . write ( \"freepythontips.wordpress.com\" ) file . close () But what if an exception occurs while writing to file ? then the file won't be closed properly. I know how much trouble it can cause. It once happened with me that i was running a web scrapper and it was running for past three hours and then unexpectedly an exception occured and my whole csv file got corrupted and i had to run the scrapper again. So lets come to the point. What method can you adopt to prevent such disaster. Obviously you can not prevent the exception when it is unexpected but you can take some precautions. Firstly you can wrap up your code in a try-except clause or better yet you can use a with statement. Let's first talk about try-except clause. You would normally do something like this : try: file = open('file.txt','w') file.write(data) except: # do something in case of an exception finally: # here comes the trick. The finally clause gets # executed even if an exception occured in try # or except clause. now we can safely close this # file. file.close() Now lets talk about the with statement. While using a with statement you would normally do: with open ( 'file.txt' , 'w' ) as file: file . write ( data ) Okay so here while using the with statement we do not have to close the file explicitly. It is automatically closed after the data is written to the file. Do share your views about this post below in the comments.","tags":"python","loc":"http://yasoob.github.io/blog/the-with-statement","title":"The with statement"},{"text":"I guess it's not a simple python server. It is really simple and easy to deploy python server. Just imagine you want to share a folder on your pc with someone on your network. What you will do ? Obviously you will start the sharing service but what if you have python installed and want an easier solution ? Here you go. You can just type in : $ python - m SimpleHTTPServer This should be typed in the terminal or command prompt if you are a windows guy. Just open command prompt. Navigate to the directory which you want to share and then run this command. Unlike the normal sharing services this server will also show you the IP addresses of the PCs which are accessing that folder. Pretty slick right ? Do share your views about this post below in the comments.","tags":"python, server","loc":"http://yasoob.github.io/blog/a-simple-python-server","title":"A simple python server"},{"text":"Okay so this is kinda new for some of us. If you are a seasoned programmer then this might not be surprising for you but if you are a new programmer then this is something you really need to know. Okay so here we go. In normal situations if you would want to swap the values of two variables then this would be something you would go after : a = 1 b = 2 c = a a = b b = c But there's a more simple method which will merely take three lines. Here it is : a = 1 b = 2 a , b = b , a This is surely a simple way to swap values. Do comment below and tell me your views about this.","tags":"python","loc":"http://yasoob.github.io/blog/in-place-value-swapping","title":"In-place value swapping"},{"text":"Hi there, Today i am beginning my very own personal blog related to python. Let me first tell you who i am. I am a computer programmer from Pakistan. I have made a lot open source contributions. I have contributed mainly to youtube-dl. It's a python script which allows you to download videos from various websites like youtube, facebook, dailymotion, metacafe, etc . I also run a website with the name of http://www.soundcloud-dl.appspot.com/ It allows you to search and download music from soundcloud.com . I know that sounds cool :D . I have done all of it's backend programming in python. I also have some useful scripts on my github page and if you would like to check them out then you are most welcome. This blog will serve you new python tips every other day. This blog is not for seasoned programmers but those who are starting out with python or want to enhance their knowledge about this language. lastly if you have any comments, suggestions or want my help with something then feel free to email me on yasoob.khld@gmail.com . I guess now it's time to say goodbye.","tags":"welcome","loc":"http://yasoob.github.io/blog/welcome","title":"The beginning of a new blog."}]}