Python Misc

https://stackoverflow.com/questions/11517907/howto-add-documentation-comment-in-pycharm-ide

After typing the function heading (e.g., def myfunc(...)) and the following colon (:) press enter, then type """ (3 double-quotes), and press enter again.

Alt+Enter (activate intention action), Insert documentation string stub.

http://massivetechinterview.blogspot.com/2018/03/python-misc-2.html

os.chdir("../..")
Don't change current directory...
Don't change state, hard to test, bug

https://henrikwarne.com/2014/06/22/switching-from-java-to-python-first-impressions/
Tuples
Returning multiple values.

max, min, avg = summarize(source) Functions as arguments

Stand-alone functions
Compact code.

BAD

No Static Types (1)

No Static Types (2). The other consequence of the dynamic typing in Python is that the “find usage” feature in the IDE is not as useful. I am using PyCharm for Python, and used IntelliJ IDEA for Java. In Java, I used “find usage” a lot to find out where a method is called from, where a class is used etc. In PyCharm the accuracy is (naturally) worse – often there are many false positives for usage, unless the name of the function is unique. So it is basically back to grep:ing the code base for a given string (like in my C++ days).

pip install virtualenv

Setup virtualenv for this repo

mkdir ~/virtualenv && virtualenv -p python2.7 ~/virtualenv/repo_name

Enable virtualenv

source ~/virtualenv/repo_name/bin/activate

https://stackoverflow.com/questions/41573587/what-is-the-difference-between-venv-pyvenv-pyenv-virtualenv-virtualenvwrappe
pyenv is used to isolate Python versions. For example, you may want to test your code against Python 2.6, 2.7, 3.3, 3.4 and 3.5, so you'll need a way to switch between them.

• pyvenv is a script shipped with Python 3 but deprecated in Python 3.6 as it had problems (not to mention the confusing name). In Python 3.6+, the exact equivalent is python3 -m venv.

• pipenv, by Kenneth Reitz (the author of requests), is the newest project in this list. It aims to combine Pipfile, pip and virtualenv into one command on the command-line.

I would just avoid the use of virtualenv after Python3.3+ and instead use the standard shipped library venv. To create a new virtual environment you would type:

$ python3 -m venv <MYVENV>  

virtualenv tries to copy the Python binary into the virtual environment's bin directory. However it does not update library file links embedded into that binary, so if you build Python from source into a non-system directory with relative path names, the Python binary breaks. Since this is how you make a copy distributable Python, it is a big flaw

https://github.com/pypa/pipenv

• To initialize a Python 3 virtual environment, run $ pipenv --three.
• To initialize a Python 2 virtual environment, run $ pipenv --two.

https://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/

pip is a tool for installing Python packages from the Python Package Index.

PyPI (which you'll occasionally see referred to as The Cheeseshop) is a repository for open-source third-party Python packages. It's similar to RubyGems in the Ruby world, PHP's Packagist, CPAN for Perl, and NPM for Node.js.

Python actually has another, more primitive, package manager called easy_install, which is installed automatically when you install Python itself. pip is vastly superior to easy_install for lots of reasons, and so should generally be used instead. You can use easy_install to install pip as follows:

1	$ sudo easy_install pip

in most cases, you shouldn't install packages globally

virtualenv solves a very specific problem: it allows multiple Python projects that have different (and often conflicting) requirements, to coexist on the same computer.

virtualenv solves this problem by creating a completely isolated virtual environment for each of your programs. An environment is simply a directory that contains a complete copy of everything needed to run a Python program, including a copy of the python binary itself, a copy of the entire Python standard library, a copy of the pip installer, and (crucially) a copy of the site-packages directory mentioned above. When you install a package from PyPI using the copy of pip that's created by the virtualenv tool, it will install the package into the site-packages directory inside the virtualenv directory. 

sudo pip install virtualenv. Usually pip and virtualenv are the only two packages you ever need to install globally, because once you've got both of these you can do all your work inside virtual environments.

sudo pip install --upgrade requests

virtualenv env

env is just the name of the directory you want to create your virtual environment inside

Instead of typing env/bin/python and env/bin/pipevery time, we can run a script to activate the environment. This script, which can be executed with source env/bin/activate, simply adjusts a few variables in your shell (temporarily) so that when you type python, you actually get the Python binary inside the virtualenv instead of the global one

The adjustments to your shell only last for as long as the terminal is open

virtualenv and pip make great companions, especially when you use the requirements feature of pip. Each project you work on has its own requirements.txt file, and you can use this to install the dependencies for that project into its virtual environment:

1	env/bin/pip install -r requirements.txt

https://code.tutsplus.com/tutorials/understanding-virtual-environments-in-python--cms-28272

Once you are done with the virtual environment, you can deactivate it using the following command:

1	deactivate

pip install -r requirements.txt

virtualenv --version

http://docs.python-guide.org/en/latest/dev/virtualenvs/



docstring
https://www.jetbrains.com/help/pycharm/using-docstrings-to-specify-types.html

Important note about reStructuredText/Sphinx

Note that for the reStructuredText it's possible to specify types in two formats:

• :param param_type param_name: parameter description (type description is on the same line as the parameter description).
• :type param_name: param_type (type description is on a separate line)

https://www.jetbrains.com/help/pycharm/settings-tools-python-integrated-tools.html

https://stackoverflow.com/questions/419163/what-does-if-name-main-do

When the Python interpreter reads a source file, it executes all of the code found in it.

Before executing the code, it will define a few special variables. For example, if the python interpreter is running that module (the source file) as the main program, it sets the special __name__ variable to have a value "__main__". If this file is being imported from another module, __name__ will be set to the module's name.

In the case of your script, let's assume that it's executing as the main function, e.g. you said something like

python threading_example.py

on the command line. After setting up the special variables, it will execute the import statement and load those modules. It will then evaluate the def block, creating a function object and creating a variable called myfunction that points to the function object. It will then read the if statement and see that __name__ does equal "__main__", so it will execute the block shown there.

One reason for doing this is that sometimes you write a module (a .py file) where it can be executed directly. Alternatively, it can also be imported and used in another module. By doing the main check, you can have that code only execute when you want to run the module as a program and not have it execute when someone just wants to import your module and call your functions themselves.

https://docs.python.org/3/library/__main__.html

A module can discover whether or not it is running in the main scope by checking its own __name__, which allows a common idiom for conditionally executing code in a module when it is run as a script or with python -m but not when it is imported:

if __name__ == "__main__":
    # execute only if run as a script
    main()

For a package, the same effect can be achieved by including a __main__.py module, the contents of which will be executed when the module is run with -m.

https://stackoverflow.com/questions/2817264/how-to-get-the-parent-dir-location

You can apply dirname repeatedly to climb higher: dirname(dirname(file)). This can only go as far as the root package, however. If this is a problem, use os.path.abspath: dirname(dirname(abspath(file))).

http://doc.pytest.org/en/latest/getting-started.html

1. Run the following command in your command line:

   pip install -U pytest
   

2. Check that you installed the correct version:

   $ pytest --version

# content of test_sample.py
def func(x):
    return x + 1

def test_answer():
    assert func(3) == 5

https://stackoverflow.com/questions/36456920/is-there-a-way-to-specify-which-pytest-tests-to-run-from-a-file

You can use -k option to run test cases with different patterns:

py.test tests_directory/foo.py tests_directory/bar.py -k 'test_001 or test_some_other_test'

This will run test cases with name test_001 and test_some_other_test deselecting the rest of the test cases.

Note: This will select any test case starting with test_001 or test_some_other_test. For example, if you have test case test_0012 it will also be selected.

https://github.com/pypa/pip/issues/3761

You don't have rw permission to the directories. Assuming you have administrator rights, you can change the permissions with sudo chown -R $USER /absolute/path/to/directory. If you're running OS X 10.11+, you'll probably also need disable the System Integrity Protection, but don't forget to reenable it after you're done (just to be safe).

@tallakahath sudo chown -R $USER /usr/local/lib/python2.7/
@JikkuJose sudo chown -R $USER /Library/Python/2.7

pycharm multiple thread debug issue
https://stackoverflow.com/questions/21157739/how-to-iterate-through-a-python-queue-queue-with-a-for-loop-instead-of-a-while-l

q = Queue.Queue()
for elem in list(q.queue):
    print elem

I prefer this over iter(queue.get as it does not rely on the sentinel.

https://www.daniweb.com/programming/software-development/threads/259751/function-returning-a-value-multi-thread

A couple other things: you're calling join(), which will block until the thread is finished making the while loop directly after it unnecessary, and don't use "str" as a variable name since it's already the name of a builtin function.

import threading
import Queue

def stringFunction(value, out_queue):
    my_str = "This is string no. " + value
    out_queue.put(my_str)

my_queue = Queue.Queue()
thread1 = threading.Thread(stringFunction("one", my_queue))
thread1.start()
thread1.join()

func_value = my_queue.get()
print func_value

The tread must also be created by

thread1 = threading.Thread(target = stringFunction, arg = ("one", my_queue))

and not threading.Thread(stringFunction("one", my_queue)).

http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm

The Global Interpreter Lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via sys.setcheckinterval. Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python program.

https://opensource.com/article/17/4/grok-gil

https://christopherdavis.me/blog/threading-basics.html

python 3

The python queue module implements a set of queues that are safe for multi-threaded environments. We don't want to use something like a list to share between threads because its not thread safe. In other words, two threads could call pop at the same time and get the same value.

https://mirrors.segmentfault.com/python2/queue.html

>>> add = lambda x, y: x + y
>>> add(2,3)




https://stackoverflow.com/questions/2970858/why-doesnt-print-work-in-a-lambda

Why doesn't this work?

lambda: print "x"

A lambda's body has to be a single expression. In Python 2.x, print is a statement. However, in Python 3, print is a function (and a function application is an expression, so it will work in a lambda). You can (and should, for forward compatibility :) use the back-ported print function if you are using the latest Python 2.x:

In [1324]: from __future__ import print_function

In [1325]: f = lambda x: print(x)

In [1326]: f("HI") 

https://stackoverflow.com/questions/35244577/is-it-possible-to-use-an-inline-function-in-a-thread-call

You can use a lambda function in Python 3.x

import threading

threading.Thread(target=lambda a: print("Hello, {}".format(a)), args=(["world"]))

from __future__ import print_function # I'm on Python 2.7
from threading import Thread

Thread(target=(lambda: print('test') == print('hello'))).start()




Threading

https://stackoverflow.com/questions/11968689/python-multithreading-wait-till-all-threads-finished

t1 = Thread(target=call_script, args=(scriptA + argumentsA))
t2 = Thread(target=call_script, args=(scriptA + argumentsB))
t3 = Thread(target=call_script, args=(scriptA + argumentsC))

t1.start()
t2.start()
t3.start()

t1.join()
t2.join()
t3.join()




https://pymotw.com/2/threading/

import threading

threads = []
for i in range(5):
    t = threading.Thread(target=worker)
    threads.append(t)
    t.start()

https://stackoverflow.com/questions/5615648/python-call-function-within-class
Python call function within class

Since these are member functions, call it as a member function on the instance, self.

def isNear(self, p):
    self.distToPoint(p)

https://www.programiz.com/python-programming/string-interpolation

f-strings

Python 3.6 added new string interpolation method called literal string interpolation and introduced a new literal prefix f.

print(f'Hello {name}! This is {program}')

This new string interpolation is powerful as we can embed arbitrary Python expressions we can even do inline arithmetic with it.

print(f'12 multiply 3 is {a * b}.')




%-formatting

print("%s %s" %('Hello','World',))

as the % operator only takes one argument, we need to wrap the right-hand side in a tuple

It’s also possible to refer to variable substitutions by name in our format string, if we pass a mapping to the % operator:

print(‘Hello %(name)s! This is %(program)s.’%(name,program))

Str.format()

In this string formatting we use format() function on a string object and braces {}, the string object in format() function is substituted in place of braces {}. We can use the format() function to do simple positional formatting, just like % formatting.

print('Hello, {}'.format(name))




print('Hello {name}!This is{program}.'.format(name=name,program=program))




from string import Template
name = 'world'
program ='python'
new = Template('Hello $name! This is $program.')
print(new.substitute(name= name,program=program))




1. %-format method is very old method for interpolation and is not recommended to use as it decrease the code readability.
2. In str.format() method we pass the string object to the format() function for string interpolation.
3. In template method we make a template by importing template class from built in string module.
4. Literal String Interpolation method is powerful interpolation method which is easy to use and increase the code readability.

Static class variables in Python
https://stackoverflow.com/questions/68645/static-class-variables-in-python

Variables declared inside the class definition, but not inside a method are class or static variables:

>>> class MyClass:
...     i = 3
...
>>> MyClass.i
3 

As @millerdev points out, this creates a class-level i variable, but this is distinct from any instance-level i variable, so you could have

>>> m = MyClass()
>>> m.i = 4
>>> MyClass.i, m.i
>>> (3, 4)

This is different from C++ and Java, but not so different from C#, where a static member can't be accessed using a reference to an instance.

TypeError: cannot concatenate 'str' and 'dict' objects
https://stackoverflow.com/questions/26632186/typeerror-cannot-concatenate-str-and-dict-objects

print "web_response=", web_response

Determining the path that a yum package installed to
https://stackoverflow.com/questions/1766380/determining-the-path-that-a-yum-package-installed-to

yum uses RPM, so the following command will list the contents of the installed package:

$ rpm -ql package-name

https://gist.github.com/dalegaspi/dec44117fa5e7597a559

sudo yum install python27 # install Python 2.7

rpm -ql python2.7-stack-2.7.12-1.9.g680e0d1.x86_64

/opt/python-2.7/bin/python -V

debug
https://pypi.python.org/pypi/pdbpp/

https://pypi.python.org/pypi/ipdb
http://www.christianlong.com/blog/ipdb-the-ipython-debugger.html
python2.7 -m pip install pudb

python2.7 -m pudb.run s.py

https://github.com/sagemathinc/cocalc/wiki/Using-a-GUI-Python-Debugger-with-Sage-(PUDB)

• n: Execute next command
• s: Step into a function
• c: Continue execution
• b: Set a breakpoint on the current line
• e: Show the traceback from a thrown exception
• q: Opens dialog to either quit or to restart the running program
• o: Show the original console / standard output screen
• m: Open a module in a different file
• L: Go to line
• !: Go to the Python command line subwindow at the bottom of the screen
• ?: Display the help dialog which includes a complete listing of shortcut commands
• <SHIFT+V>: Switch context to the variables subwindow on the right of the screen
• <SHIFT+B>: Switch context to the breakpoints subwindow on the right of the screen
• <CTRL+X>: Toggle contexts between the lines of code and the Python command line

Others
python2.7 -m pip install ipdb

python2.7 -m pip install pdbpp

https://passingcuriosity.com/2015/installing-python-from-source/

./configure --prefix=$HOME/py-351
make
make test
make install

make altinstall

sudo yum -y groupinstall "Development Tools"

sudo yum -y install gcc gcc-c++

wget https://www.python.org/ftp/python/2.7.14/Python-2.7.14.tgz

tar xzf Python-2.7.14.tgz

cd Python-2.7.14

./configure --enable-optimizations

make install

#altinstall

python2.7 -m ensurepip --default-pip

sudo yum -y install python-devel libffi-devel openssl-devel

python2.7  -m pip install  requests[security] --user

https://stackoverflow.com/questions/2988017/string-comparison-in-python-is-vs

For all built-in Python objects (like strings, lists, dicts, functions, etc.), if x is y, then x==y is also True.

Not always. NaN is a counterexample. But usually, identity (is) implies equality (==). The converse is not true: Two distinct objects can have the same value.

Also, is it generally considered better to just use '==' by default, even when comparing int or Boolean values?

You use == when comparing values and is when comparing identities.

When comparing ints (or immutable types in general), you pretty much always want the former. There's an optimization that allows small integers to be compared with is, but don't rely on it.

https://stackoverflow.com/questions/1471994/what-is-setup-py

setup.py is a python file, which usually tells you that the module/package you are about to install has been packaged and distributed with Distutils, which is the standard for distributing Python Modules.

This allows you to easily install Python packages. Often it's enough to write:

$ python setup.py install

and the module will install itself.

https://google.github.io/styleguide/pyguide.html

Python evaluates certain values as false when in a boolean context. A quick "rule of thumb" is that all "empty" values are considered false so 0, None, [], {}, '' all evaluate as false in a boolean context.

https://stackoverflow.com/questions/2052390/manually-raising-throwing-an-exception-in-python

raise ValueError('A very specific bad thing happened.')

Don't raise generic exceptions

Instead, use the most specific Exception constructor that semantically fits your issue.

raise ValueError('A very specific bad thing happened')




Since Python 2.6.X you might want to use:

"my {0} string: {1}".format("cool", "Hello there!")

https://stackoverflow.com/questions/1575779/python-datetime-subtraction-wrong-results

timedelta.seconds isn't the total number of seconds, it's the remainder in seconds after days have been accounted for. Using your example:

>>> import datetime
>>> (datetime.datetime(2008,11,7,10,5,14)-datetime.datetime(2008,11,6,9,30,16))
datetime.timedelta(1, 2098)

That datetime.timedelta(1, 2098) means that your timedelta is 1 day, plus 2098 seconds.

What you want is something like:

>>> delta = (datetime.datetime(2008,11,7,10,5,14)-datetime.datetime(2008,11,6,9,30,16))
>>> (delta.days * 86400) + delta.seconds

https://eli.thegreenplace.net/2011/05/15/understanding-unboundlocalerror-in-python

x = 10
def foo():
    x += 1
    print x
foo()

So where does the exception come from? Quoting the FAQ:

This is because when you make an assignment to a variable in a scope, that variable becomes local to that scope and shadows any similarly named variable in the outer scope.

But x += 1 is similar to x = x + 1, so it should first read x, perform the addition and then assign back to x. As mentioned in the quote above, Python considers x a variable local to foo, so we have a problem - a variable is read (referenced) before it's been assigned. Python raises the UnboundLocalError exception in this case [1].

https://stackoverflow.com/questions/37651721/why-am-i-getting-a-local-variable-referenced-before-assignment-error

https://stackoverflow.com/questions/423379/using-global-variables-in-a-function-other-than-the-one-that-created-them

You can use a global variable in other functions by declaring it as global in each function that assigns to it:

globvar = 0

def set_globvar_to_one():
    global globvar    # Needed to modify global copy of globvar
    globvar = 1

def print_globvar():
    print(globvar)     # No need for global declaration to read value of globvar

set_globvar_to_one()
print_globvar()       # Prints 1

I imagine the reason for it is that, since global variables are so dangerous, Python wants to make sure that you really know that's what you're playing with by explicitly requiring the global keyword.

https://www.python-course.eu/python3_global_vs_local_variables.php
Python deals with variables the other way around. They are local, if not otherwise declared. The driving reason behind this approach is that global variables are generally bad practice and should be avoided. In most cases where you are tempted to use a global variable, it is better to utilize a parameter for getting a value into a function or return a value to get it out

https://intellij-support.jetbrains.com/hc/en-us/community/posts/207133805-Problem-with-quick-documentation-on-mouse-move

And in newer versions through settings File | Settings | Editor | General

https://stackoverflow.com/questions/7225900/how-to-pip-install-packages-according-to-requirements-txt-from-a-local-directory

$ pip install -r requirements.txt --no-index --find-links file:///tmp/packages

--no-index - Ignore package index (only looking at --find-links URLs instead).

-f, --find-links <URL> - If a URL or path to an html file, then parse for links to archives. If a local path or file:// URL that's a directory, then look for archives in the directory listing.

pip install -r /path/to/requirements.txt

https://stackoverflow.com/questions/16584552/how-to-state-in-requirements-txt-a-direct-github-source

“Editable” packages syntax can be used in requirements.txt to import packages from a variety of VCS (git, hg, bzr, svn):

-e git://github.com/mozilla/elasticutils.git#egg=elasticutils

Also, it is possible to point to particular commit:

-e git://github.com/mozilla/elasticutils.git@000b14389171a9f0d7d713466b32bc649b0bed8e#egg=elasticutils

I got somewhat confused by the -e flag in the other answers so here's my clarification:

The -e or --editable flag means that the package is installed in <venv path>/src/SomeProject and thus not in the deeply buried <venv path>/lib/pythonX.X/site-packages/SomeProject it would otherwise be placed in.²

You have to start the URL with svn+ (git+, hg+ or bzr+), and you have to include #egg=Package so pip knows what to expect at that URL. You can also include @rev in the URL, e.g., @275 to check out revision 275.

^{https://pip.readthedocs.io/en/1.1/requirements.html#requirements-file-format}

https://www.jetbrains.com/help/pycharm/creating-requirement-files.html

1. Open the Settings/Preferences dialog box, and then click the page Python Integrated Tools.
2. In the Package requirements file field, type the name of the requirements file. By default, requirements.txt is suggested. You can either accept default, or click the browse button and locate the desired file

https://stackoverflow.com/questions/31649390/python-requests-ssl-handshake-failure

_ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

I only had to to install requestsusing its security setup:

pip install requests[security]

This installs pyOpenSSL, ndg-httpsclient and pyasn1.https://github.com/kennethreitz/requests/blob/master/setup.py#L70

https://support.dominodatalab.com/hc/en-us/articles/115001152486-Why-do-I-get-a-permission-denied-error-when-I-try-to-pip-install-packages-in-a-workspace-session-

!pip install --upgrade --user scikit-learn

However, this installation will not persist from run to run. If you consistently need this library upgrade, you have a couple of options:

• You can add scikit-learn==0.18.1 to your project's requirements.txt file, as outlined here: http://support.dominodatalab.com/hc/en-us/articles/204842945-How-can-I-install-packages-or-dependencies-#python . This will install it automatically at the start of every run.

pip install --user jupyterlab to avoid putting it in the system location?
https://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-and-requests-module

urllib2 provides some extra functionality, namely the urlopen() function can allow you to specify headers (normally you'd have had to use httplib in the past, which is far more verbose.) More importantly though, urllib2 provides the Request class, which allows for a more declarative approach to doing a request:

r = Request(url='http://www.mysite.com')
r.add_header('User-Agent', 'awesome fetcher')
r.add_data(urllib.urlencode({'foo': 'bar'})
response = urlopen(r)

Note that urlencode() is only in urllib, not urllib2.

There are also handlers for implementing more advanced URL support in urllib2. The short answer is, unless you're working with legacy code, you probably want to use the URL opener from urllib2, but you still need to import into urllib for some of the utility functions.

import ssl
print ssl.OPENSSL_VERSION

https://docs.python.org/2/reference/datamodel.html

https://stackoverflow.com/questions/25384196/importing-projects-into-eclipse-python
http://www.pydev.org/faq.html#PyDevFAQ-HowdoIimportexistingprojects%2FsourcesintoPyDev%3F

How do I import existing projects/sources into PyDev?

If the project was already in Eclipse (i.e.: has the .project and .pydevproject files), use File > Import > Existing Projects into Workspace.

Now, if the project never had the .project nor .pydevproject files, there are 2 choices:

Option 1

Just go through the File > New > PyDev Project wizard and select as the directory the directory with your sources (properly configuring the source folders in the process -- see: Creating a Project for more information).

Note on source folders: If the file structure you have is:

myproject
.../ _init_.py
.../ module.py

and you want to make the import:

from myproject import module

you have to set as the source folder the folder above myproject (as that same folder, above myproject, will be added to the PYTHONPATH -- i.e.: source folders are the folders added to the PYTHONPATH).

Note on links: links can be used above the source folder or as the source folder, but never inside a source folder, as it's not supported by PyDev.

Atom's python-tools go-to definition function doesn't work for custom code.
https://docs.python.org/3/library/argparse.html

• 'store_const' - This stores the value specified by the const keyword argument. The 'store_const' action is most commonly used with optional arguments that specify some sort of flag. For example:
  >>>
  

  >>> parser = argparse.ArgumentParser()
  >>> parser.add_argument('--foo', action='store_const', const=42)
  >>> parser.parse_args(['--foo'])
  Namespace(foo=42)

• 'append' - This stores a list, and appends each argument value to the list. This is useful to allow an option to be specified multiple times. Example usage:
  >>>
  

  >>> parser = argparse.ArgumentParser()
  >>> parser.add_argument('--foo', action='append')
  >>> parser.parse_args('--foo 1 --foo 2'.split())
  Namespace(foo=['1', '2'])

>>> i = 123
>>> type(i)
<type 'int'>

https://docs.python.org/2/library/argparse.html

In a script, parse_args() will typically be called with no arguments, and the ArgumentParser will automatically determine the command-line arguments from sys.argv.

python if __name__ == "__main__"
https://docs.python.org/3/library/__main__.html

'__main__' is the name of the scope in which top-level code executes. A module’s __name__ is set equal to '__main__' when read from standard input, a script, or from an interactive prompt.

A module can discover whether or not it is running in the main scope by checking its own __name__, which allows a common idiom for conditionally executing code in a module when it is run as a script or with python -m but not when it is imported:

if __name__ == "__main__":
    # execute only if run as a script
    main()

For a package, the same effect can be achieved by including a __main__.py module, the contents of which will be executed when the module is run with -m.

http://www.johnny-lin.com/cdat_tips/tips_pylang/path.html

Your path (i.e. the list of directories Python goes through to search for modules and files) is stored in the path attribute of the sys module. Since path is a list, you can use the append method to add new directories to the path.

For instance, to add the directory /home/me/mypy to the path, just do:

import sys
sys.path.append("/home/me/mypy")

sys.stderr.write
http://www.geeksforgeeks.org/how-to-write-an-empty-function-in-python-pass-statement/

In Python, to write empty functions, we use pass statement. pass is a special statement in Python that does nothing. It only works as a dummy statement.

def fun():

    pass

http://www.geeksforgeeks.org/str-vs-repr-in-python/

• str() is used for creating output for end user while repr() is mainly used for debugging and development. repr’s goal is to be unambiguous and str’s is to be readable. For example, if we suspect a float has a small rounding error, repr will show us while str may not.
• repr() compute the “official” string representation of an object (a representation that has all information about the abject) and str() is used to compute the “informal” string representation of an object (a representation that is useful for printing the object).
• The print statement and str() built-in function uses __str__ to display the string representation of the object while the repr() built-in function uses __repr__ to display the object.

http://www.tutorialspoint.com/python/python_basic_syntax.htm
#!/usr/bin/python
print "Hello, Python!"
https://docs.python.org/2/library/functions.html
http://www.sdlczone.com/python-sum-function/
sum(iterable[, start]) :
This function sums start and the items of an iterable from left to right and returns the total. start defaults to 0. The iterable‘s items are normally numbers, and the start value is not allowed to be a string.
For some use cases, there are good alternatives to sum(). The preferred, fast way to concatenate a sequence of strings is by calling ”.join(sequence). To add floating point values with extended precision, see math.fsum(). To concatenate a series of iterables, consider using itertools.chain().

>>> x=[1,2,3,4,5,6,7,8]

>>> sum(x)

36

>>> sum(x,2)

38

https://pymotw.com/2/collections/counter.html

A Counter is a container that keeps track of how many times equivalent values are added. It can be used to implement the same algorithms for which bag or multiset data structures are commonly used in other languages.

import collections

c = collections.Counter()
print 'Initial :', c

c.update('abcdaab')
print 'Sequence:', c

c.update({'a':1, 'd':5})
print 'Dict    :', c

The count values are increased based on the new data, rather than replaced. 

The elements() method returns an iterator that produces all of the items known to the Counter.

Use most_common() to produce a sequence of the n most frequently encountered input values and their respective counts.

for letter, count in c.most_common(3):
    print '%s: %7d' % (letter, count)

http://www.saltycrane.com/blog/2007/09/how-to-sort-python-dictionary-by-keys/

Starting with Python 2.4, both list.sort() and sorted() added a key parameter to specify a function to be called on each list element prior to making comparisons.

sorted("This is a test string from Andrew".split(), key=str.lower)

sorted(student_tuples, key=lambda student: student[2])   # sort by age

sorted(student_objects, key=lambda student: student.age)   # sort by age

The method items() returns a list of dict's (key, value) tuple pairs

https://www.accelebrate.com/blog/using-defaultdict-python/

defaultdict means that if an key is not found in the dictionary that instead of a KeyError being thrown a new value is created. The type of this new pair is given by the argument of defaultdict.

It gets more interesting when the values in a dictionary are collections (lists, dicts, etc.) In this case, the value (an empty list or dict) must be initialized the first time a given key is used. While this is relatively easy to do manually, the defaultdict type automates and simplifies these kinds of operations.

A defaultdict works exactly like a normal dict, but it is initialized with a function (“default factory”) that takes no arguments and provides the default value for a nonexistent key.

A defaultdict will never raise a KeyError. Any key that does not exist gets the value returned by the default factory.

>>> from collections import defaultdict
>>> ice_cream = defaultdict(lambda: 'Vanilla')
>>> ice_cream['Sarah'] = 'Chunky Monkey'
>>> print ice_cream['Joe']
Vanilla

The default factory is int, which in turn has a default value of zero. (Note: “lambda: 0″ would also work in this situation). For each food in the list, the value is incremented by one where the key is the food. We do not need to make sure the food is already a key – it will use the default value of zero.

>>> food_count = defaultdict(int) # default value of int is 0
>>> for food in food_list:
...     food_count[food] += 1 # increment element's value by 1

To build this dictionary of lists, we use a defaultdict with a default factory of list. A new list is created for each new key.

>>> cities_by_state = defaultdict(list)
>>> for state, city in city_list:
...     cities_by_state[state].append(city)

http://learnpythonthehardway.org/book/ex39.html

>>> stuff = {'name': 'Zed', 'age': 39, 'height': 6 * 12 + 2}
>>> print stuff['name']

>>> stuff[1] = "Wow"
>>> stuff[2] = "Neato"
>>> print stuff[1]

map function
https://docs.python.org/2/library/functions.html#map

map(function, iterable, ...)

Apply function to every item of iterable and return a list of the results. If additional iterable arguments are passed,function must take that many arguments and is applied to the items from all iterables in parallel. If one iterable is shorter than another it is assumed to be extended with None items. If function is None, the identity function is assumed; if there are multiple arguments, map() returns a list consisting of tuples containing the corresponding items from all iterables (a kind of transpose operation). The iterable arguments may be a sequence or any iterable object; the result is always a list.

>>> def upper(s):
...     return s.upper()

map(upper, ['sentence', 'fragment'])

http://www.bogotobogo.com/python/python_fncs_map_filter_reduce.php

list(map((lambda x: x **2), items))

def square(x):
        return (x**2)
def cube(x):
        return (x**3)

funcs = [square, cube]
for r in range(5):
    value = map(lambda x: x(r), funcs)
    print value

https://azitech.wordpress.com/2011/07/06/pythons-enumerate-function/
enumerate() function: it allows you to create a for loop where you’re iterating over a sequence (list, tuple, etc) and have both the item and its index in the loop.

for ind, itm in enumerate(x):
    print("x["+str(ind)+"] = "+str(itm))

how to python prettyprint a json file

echo '{"foo": "bar"}' | python -m json.tool | pygmentize -g

PYTHON救急HTTPSERVER和FTPSERVER
https://hg.python.org/cpython/file/2.7/Lib/BaseHTTPServer.py
python有个-m参数可以运行一些现有的内置的模块，当然模块可以自己pip install module_name安装。
python -m SimpleHTTPServer 8000
Python 2.x
python -m SimpleHTTPServer [port]
Python 3.x
python -m http.server [port]

PYTHON 简易FTPSERVER
Python没有内置一个直接可以用的FTP服务器，所以需要第三方组件的支持，我找到的这个组件叫pyftpdlib，首先安装：
pip install pyftpdlib
python -m pyftpdlib -p 21
是匿名访问，也就是用户名是anonymous，密码为空

https://docs.python.org/3/library/operator.html
10.3.1. Mapping Operators to Functions

Containment Test	obj in seq	contains(seq, obj)
Division	a / b	truediv(a, b)
Division	a // b	floordiv(a, b)

Exponentiation	a ** b	pow(a, b)
Identity	a is b	is_(a, b)
Identity	a is not b	is_not(a, b)

Automating File Movement on your system

def OrganizeDirectory(sourcePath, extensionToDir):

    if not os.path.exists(sourcePath):

        print ("The source folder '" + sourcePath +

              "' does not exist!!\n")

    else:

        for file in os.listdir(sourcePath):

            file = os.path.join(sourcePath, file)

            # Ignore if its a directory

            if os.path.isdir(file):

                continue

            filename, fileExtension = os.path.splitext(file)

            fileExtension = fileExtension[1:]

            # If the file extension is present in the mapping

            if fileExtension in extensionToDir:

                # Store the corresponding directory name

                destinationName = extensionToDir[fileExtension]

                destinationPath = os.path.join(sourcePath, destinationName)

                # If the directory does not exist

                if not os.path.exists(destinationPath):

                    print ("Creating new directory for `" + fileExtension +

                          "` files, named - `" + destinationName + "'!!")

                    # Create a new directory

                    os.makedirs(destinationPath)

                # Move the file

                shutil.move(file, destinationPath)

http://www.thegeekstuff.com/2014/06/python-sorted/
  >>> l = [3, 2, 5 ,4, 7, 1]
  >>> sorted(l)
  >>> t = ('Zane', 'Bob', 'Janet')
  >>> sorted(t)
  >>> d = {1:'a', 2:'b', 3:'c'}
  >>> sorted(d)
  [1, 2, 3]
every time, the sorted() function returns a list, even if that is not the type that was passed in. In the dictionary case, it returns a sorted list of the dictionaries keys.

2. Sorting Complex Structures using a Key
The sorted() function accepts a key as an optional named parameter.
This key should be, itself, a function which accepts one parameter and will be used by the sorted() function to determine a value to sort on.

  >>> class Person(object):
  ...     def __init__(self, name, age):
  ...             self.name = name
  ...             self.age = age
  ...     def __repr__(self):
  ...             return "<name: %s, age: %s>" % (self.name, self.age)
(The __repr__ function is a special function used to override how the object will be represented when displayed by the python interpreter.
By itself, sorted() doesn’t know what to do with the list of people:

  >>> sorted(people)
  [<name: Jack, age: 19>, <name: Adam, age: 43>, <name: Becky, age: 11>]
However, we can tell the sorted() function which attribute to sort on by specifying a key to be used. Let’s define one:

  >>> def byName_key(person):
  ...     return person.name
  ...
A key function must accept a single argument and returns a value on which to base the sorting (sorted() will call the key function on each element of the iterable it is given, and use this return value when sorting the list.)

  >>> sorted(people, key = byName_key)
sorted(people, key = byAge_key)

3. Reverse Sorting
The function accepts an optional, named parameter ‘reverse’ which should be a boolean.
  >>> sorted(l, reverse = True)

4. Sort using attrgetter Function
the standard library provides a function that can generate key functions for you:
  >>> from operator import attrgetter

We specify the name of the attribute to be fetched, and attrgetter generates a function that accepts an object and return the specified attribute from that object.
  >>> getName = attrgetter('name')
  >>> getName(jack)
So attrgetter(‘name’) returns a function which behaves like our previously defined byName_key():

  >>> sorted(people, key = attrgetter('name'))

6. Random Sorting
  >>> from random import random
  >>> def randomOrder_key(element):
  ...     return random()
  >>> sorted(snakes, key = randomOrder_key)
http://www.geeksforgeeks.org/g-fact-21-increment-and-decrement-operators-in-python/

If you’re familiar with Python, you would have known Increment and Decrement operators ( both pre and post) are not allowed in it.

for i in range(0, 5):

   print(i)

http://www.geeksforgeeks.org/py-facts-10-interesting-facts-about-python/
2. One can return multiple values in Python. 

def func():

   return 1, 2, 3, 4, 5

3. One can use an “else” clause with a “for” loop in Python. It’s a special type of syntax that executes only if the for loop exits naturally, without any break statements.

def func(array):      for num in array:         if num%2==0:             print(num)             break # Case1: Break is called, so 'else' wouldn't be executed.      else: # Case 2: 'else' executed since break is not called         print("No call for Break. Else is executed")

5. Function Argument Unpacking is another awesome feature of Python. One can unpack a list or a dictionary as function arguments using * and ** respectively. This is commonly known as the Splat operator.

def point(x, y):

    print(x,y)

foo_list = (3, 4)

bar_dict = {'y': 3, 'x': 2}

point(*foo_list) # Unpacking Lists

point(**bar_dict) # Unpacking Dictionaries

3 4
2 3

6. Want to find the index inside a for loop? Wrap an iterable with ‘enumerate’ and it will yield the item along with its index. See this code snippet

# Know the index faster vowels=['a','e','i','o','u'] for i, letter in enumerate(vowels):     print (i, letter)

7. One can chain comparison operators in Python answer= 1<x<10 is executable in Python.

i = 5;

ans = 1 < i < 10

print(ans)

ans = 10 > i <= 9

print(ans) 

ans = 5 == i

print(ans)

9. Instead of building a list with a loop, one can build it more concisely with a list comprehension.

# Simple List Append a = [] for x in range(0,10):     a.append(x) print(a) # List Comprehension print([x for x in a])

10. Finally, Python’s special Slice Operator. It is a way to get items from lists, as well as change them. 

# Slice Operator a = [1,2,3,4,5] print(a[0:2]) # Choose elements [0-2), upper-bound noninclusive print(a[0:-1]) # Choose all but the last print(a[::-1]) # Reverse the list print(a[::2]) # Skip by 2 print(a[::-2]) # Skip by -2 from the back

http://hangar.runway7.net/python/packing-unpacking-arguments

def func1(x, y, z):
    print x
    print y 
    print z                 

def func2(*args):
    # Convert args tuple to a list so we can modify it
    args = list(args)
    args[0] = 'Hello'
    args[1] = 'awesome'
    func1(*args)

func2('Goodbye', 'cruel', 'world!')
# Will print
# > Hello
# > awesome
# > world!

This happens simply because we're changing the first two arguments before passing them off tofunc1.

The normal rules governing method definition apply here… calling func2('a', 'b', 'c', 'd') will raise an error because it will in turn call func1 with four arguments, which it doesn't expect.

http://www.saltycrane.com/blog/2008/01/how-to-use-args-and-kwargs-in-python/

def test_var_kwargs(farg, **kwargs):
    print "formal arg:", farg
    for key in kwargs:
        print "another keyword arg: %s: %s" % (key, kwargs[key])

test_var_kwargs(farg=1, myarg2="two", myarg3=3)

Results:

formal arg: 1
another keyword arg: myarg2: two
another keyword arg: myarg3: 3

def test_var_args_call(arg1, arg2, arg3):
    print "arg1:", arg1
    print "arg2:", arg2
    print "arg3:", arg3

kwargs = {"arg3": 3, "arg2": "two"}
test_var_args_call(1, **kwargs)

Results:

arg1: 1
arg2: two
arg3: 3

http://www.geeksforgeeks.org/division-operator-in-python/

The real floor division operator is “//”. It returns floor value for both integer and floating point arguments.

How about Python 3?
Here is another surprise, In Python 3, ‘/’ operator does floating point division for both int and float arguments.

# A Python 3 program to demonstrate use of 
# "/" for both integers and floating points
print (5/2)
print (-5/2)
print (5.0/2)
print (-5.0/2)

Output:

2.5
-2.5
2.5
-2.5

http://www.geeksforgeeks.org/g-fact-41-multiple-return-values-in-python/
2) Using Tuple: A Tuple is a comma separated sequence of items. It is created with or without (). Tuples are immutable.

def fun():

    str = "geeksforgeeks"

    x   = 20

    return str, x;  # Return tuple, we could also

                    # write (str, x)

str, x = fun() # Assign returned tuple

3) Using a list: A list is like an array of items created using square brackets. They are different from arrays as they can contain items of different types. Lists are different from tuples as they mutable.

def fun():

    str = "geeksforgeeks"

    x = 20  

    return [str, x]; 

4) Using a Dictionary: A Dictionary is similar to hash or map in other languages. 

def fun():

    d = dict();

    d['str'] = "GeeksforGeeks"

    d['x']   = 20

    return d

http://www.geeksforgeeks.org/use-yield-keyword-instead-return-keyword-python/

The yield statement suspends function’s execution and sends a value back to caller, but retains enough state to enable function to resume where it is left off. When resumed, the function continues execution immediately after the last yield run. This allows its code to produce a series of values over time, rather them computing them at once and sending them back like a list.

def simpleGeneratorFun():

    yield 1

    yield 2

    yield 3

Return sends a specified value back to its caller whereas Yield can produce a sequence of values. We should use yield when we want to iterate over a sequence, but don’t want to store the entire sequence in memory.

Yield are used in Python generators. A generator function is defined like a normal function, but whenever it needs to generate a value, it does so with the yield keyword rather than return. If the body of a def contains yield, the function automatically becomes a generator function.

# An infinite generator function that prints

# next square number. It starts with 1

def nextSquare():

    i = 1;

    # An Infinite loop to generate squares

    while True:

        yield i*i               

        i += 1  # Next execution resumes

                # from this point    

# Driver code to test above generator

# function

for num in nextSquare():

    if num > 100:

         break   

    print num

http://www.snarky.ca/why-print-became-a-function-in-python-3
syntax is reserved for things that are either impossible to do otherwise or there is a clear readability benefit to the syntax. In the case of print, the difference between print A and print(A) is negligible and thus there is no loss in readability. And since we were able to completely replace print with a function there was no loss in functionality, hence why we reduced the amount of syntax in Python 3 by making print a function (along with removing a bunch of other bits of syntax).
http://effbot.org/pyfaq/how-do-i-generate-random-numbers-in-python.htm

import random
print random.random()

This prints a random floating point number in the range [0, 1) (that is, between 0 and 1, including 0.0 but always smaller than 1.0).

There are also many other specialized generators in this module, such as:

• randrange(a, b) chooses an integer in the range [a, b).
• uniform(a, b) chooses a floating point number in the range [a, b).
• normalvariate(mean, sdev) samples the normal (Gaussian) distribution.

Some higher-level functions operate on sequences directly, such as:

• choice(S) chooses a random element from a given sequence (the sequence must have a known length).
• shuffle(L) shuffles a list in-place, i.e. permutes it randomly

random.randint(0,9)


Books
https://medium.com/level-up-web/best-python-books-in-2017-b064dfac287
Python Crash Course

https://github.com/MrAlex6204/Books

https://www.jetbrains.com/help/pycharm/quick-start-guide.html

If you want to concentrate on your own code, use the Step Into My Code button () - thus you'll avoid stepping into library classes.

Shortcut

Option+F8 - Evaluating expressions

quick fix:  option + return

The Code | Generate menu (⌘N