For example, when a minion host is ordered to read the following state file, will install

.. todo:: Write about Clint

It's based on the idea of models, an abstraction that makes it easier to

Documentation can be found `here <https://docs.djangoproject.com/en/1.3/#the-model-layer>`_

recommended that PyGTK is not used for new projects and existing applications

Kivy is a Python library for development of multi-touch enabled media rich applications. The aim is to allow for quick and easy interaction design and rapid prototyping, while making your code reusable and deployable.

The main resource for information is the website: http://kivy.org

it doesn't work with 3.0+ yet.

Also, there's a fork named `Pillow <http://pypi.python.org/pypi/Pillow>`_ which is easier

and clients, applications using SMTP, POP3, IMAP or SSH protocols, instant messaging and

can be used for message queuing without message broker. The basic patterns for this are:

- publish-subscribe: connects a set of publishers to a set of subscribers. This is a data

This section lists various so-called Python distributions which provide precompiled and

easy-to-install collections of scientific Python packages.

flavours, a paid for version and a completely free and open source community

licences for the paid-for version are available for academics and researchers.

HTML Scraping

Web Scraping

Web sites are written using HTML, which means that each web page is a

structured document. Sometimes it would be great to obtain some data from

them and preserve the structure while we're at it. Web sites provide

don't always provide their data in comfortable formats such as ``.csv``.

This is where web scraping comes in. Web scraping is the practice of using a

computer program to sift through a web page and gather the data that you need

in a format most useful to you while at the same time preserving the structure

of the data.

lxml and Requests

`lxml <http://lxml.de/>`_ is a pretty extensive library written for parsing

XML and HTML documents really fast. It even handles messed up tags. We will

also be using the `Requests <http://docs.python-requests.org/en/latest/>`_ module instead of the already built-in urlib2

due to improvements in speed and readability. You can easily install both

using ``pip install lxml`` and ``pip install requests``.

Lets start with the imports:

.. code-block:: python

Next we will use ``requests.get`` to retrieve the web page with our data

and parse it using the ``html`` module and save the results in ``tree``:

.. code-block:: python

``tree`` now contains the whole HTML file in a nice tree structure which

we can go over two different ways: XPath and CSSSelect. In this example, I

will focus on the former.

XPath is a way of locating information in structured documents such as

HTML or XML documents. A good introduction to XPath is on `W3Schools <http://www.w3schools.com/xpath/default.asp>`_ .

There are also various tools for obtaining the XPath of elements such as

FireBug for Firefox or if you're using Chrome you can right click an

element, choose 'Inspect element', highlight the code and then right

click again and choose 'Copy XPath'.

After a quick analysis, we see that in our page the data is contained in

two elements - one is a div with title 'buyer-name' and the other is a

span with class 'item-price':

::

Knowing this we can create the correct XPath query and use the lxml

``xpath`` function like this:

.. code-block:: python

Lets see what we got exactly:

.. code-block:: python

::

Congratulations! We have successfully scraped all the data we wanted from

a web page using lxml and Requests. We have it stored in memory as two

lists. Now we can do all sorts of cool stuff with it: we can analyze it

using Python or we can save it a file and share it with the world.

A cool idea to think about is modifying this script to iterate through

the rest of the pages of this example dataset or rewriting this

application to use threads for improved speed.

so the GIL is required to prevents multiple threads from running the same

`Special care`_ must be taken when writing C extensions to make sure you r

egister your threads with the interpreter.

Spanwing Processes