IPython Notebooks¶

You can run a cell by pressing [shift] + [Enter] or by pressing the "play" button in the menu.
You can get help on a function or object by pressing [shift] + [tab] after the opening parenthesis function(
You can also get help by executing function?

Numpy Arrays¶

Manipulating numpy arrays is an important part of doing machine learning (or, really, any type of scientific computation) in python. This will likely be review for most: we'll quickly go through some of the most important features.

import numpy as np

# Generating a random array
X = np.random.random((3, 5))  # a 3 x 5 array

print(X)

[[ 0.02045893  0.72606033  0.2499891   0.44950932  0.12590696]
 [ 0.910549    0.91402712  0.67770946  0.7247196   0.37554544]
 [ 0.84986851  0.86462199  0.67697981  0.05808235  0.00322052]]

# Accessing elements

# get a single element
print(X[0, 0])

# get a row
print(X[1])

# get a column
print(X[:, 1])

0.0204589341449
[ 0.910549    0.91402712  0.67770946  0.7247196   0.37554544]
[ 0.72606033  0.91402712  0.86462199]

# Transposing an array
print(X.T)

[[ 0.02045893  0.910549    0.84986851]
 [ 0.72606033  0.91402712  0.86462199]
 [ 0.2499891   0.67770946  0.67697981]
 [ 0.44950932  0.7247196   0.05808235]
 [ 0.12590696  0.37554544  0.00322052]]

# Turning a row vector into a column vector
y = np.linspace(0, 12, 5)
print(y)

[  0.   3.   6.   9.  12.]

# make into a column vector
print(y[:, np.newaxis])

[[  0.]
 [  3.]
 [  6.]
 [  9.]
 [ 12.]]

# getting the shape or reshaping an array
print(X.shape)
print(X.reshape(5, 3))

(3, 5)
[[ 0.02045893  0.72606033  0.2499891 ]
 [ 0.44950932  0.12590696  0.910549  ]
 [ 0.91402712  0.67770946  0.7247196 ]
 [ 0.37554544  0.84986851  0.86462199]
 [ 0.67697981  0.05808235  0.00322052]]

# indexing by an array of integers (fancy indexing)
indices = np.array([3, 1, 0])
print(indices)
X[:, indices]

[3 1 0]

array([[ 0.44950932,  0.72606033,  0.02045893],
       [ 0.7247196 ,  0.91402712,  0.910549  ],
       [ 0.05808235,  0.86462199,  0.84986851]])

There is much, much more to know, but these few operations are fundamental to what we'll do during this tutorial.

Scipy Sparse Matrices¶

We won't make very much use of these in this tutorial, but sparse matrices are very nice in some situations. In some machine learning tasks, especially those associated with textual analysis, the data may be mostly zeros. Storing all these zeros is very inefficient, and representing in a way that only contains the "non-zero" values can be much more efficient. We can create and manipulate sparse matrices as follows:

from scipy import sparse

# Create a random array with a lot of zeros
X = np.random.random((10, 5))
print(X)

[[ 0.78129166  0.83756364  0.84108895  0.72004361  0.77246298]
 [ 0.90326934  0.05654787  0.96044766  0.27842123  0.55005254]
 [ 0.73565863  0.86082467  0.36672653  0.87990352  0.48490203]
 [ 0.59897073  0.93126564  0.57195505  0.35149261  0.41238934]
 [ 0.41644688  0.10929447  0.63943734  0.28952431  0.8640937 ]
 [ 0.52646796  0.98127886  0.05755383  0.7423689   0.82937457]
 [ 0.25742531  0.05913226  0.39752793  0.46260783  0.57994717]
 [ 0.50462666  0.95827857  0.01949063  0.87376511  0.62576369]
 [ 0.98344586  0.24092994  0.39049177  0.96333499  0.75284338]
 [ 0.51727272  0.56962292  0.91466554  0.22258634  0.8520395 ]]

# set the majority of elements to zero
X[X < 0.7] = 0
print(X)

[[ 0.78129166  0.83756364  0.84108895  0.72004361  0.77246298]
 [ 0.90326934  0.          0.96044766  0.          0.        ]
 [ 0.73565863  0.86082467  0.          0.87990352  0.        ]
 [ 0.          0.93126564  0.          0.          0.        ]
 [ 0.          0.          0.          0.          0.8640937 ]
 [ 0.          0.98127886  0.          0.7423689   0.82937457]
 [ 0.          0.          0.          0.          0.        ]
 [ 0.          0.95827857  0.          0.87376511  0.        ]
 [ 0.98344586  0.          0.          0.96333499  0.75284338]
 [ 0.          0.          0.91466554  0.          0.8520395 ]]

# turn X into a csr (Compressed-Sparse-Row) matrix
X_csr = sparse.csr_matrix(X)
print(X_csr)

  (0, 0)	0.781291662687
  (0, 1)	0.83756363806
  (0, 2)	0.841088945341
  (0, 3)	0.720043607651
  (0, 4)	0.772462978553
  (1, 0)	0.903269339553
  (1, 2)	0.960447657459
  (2, 0)	0.735658630516
  (2, 1)	0.860824671718
  (2, 3)	0.879903521177
  (3, 1)	0.931265638958
  (4, 4)	0.864093698083
  (5, 1)	0.981278856248
  (5, 3)	0.742368904482
  (5, 4)	0.829374573348
  (7, 1)	0.958278571633
  (7, 3)	0.873765108802
  (8, 0)	0.9834458564
  (8, 3)	0.963334986571
  (8, 4)	0.752843382529
  (9, 2)	0.914665535999
  (9, 4)	0.852039495382

# convert the sparse matrix to a dense array
print(X_csr.toarray())

[[ 0.78129166  0.83756364  0.84108895  0.72004361  0.77246298]
 [ 0.90326934  0.          0.96044766  0.          0.        ]
 [ 0.73565863  0.86082467  0.          0.87990352  0.        ]
 [ 0.          0.93126564  0.          0.          0.        ]
 [ 0.          0.          0.          0.          0.8640937 ]
 [ 0.          0.98127886  0.          0.7423689   0.82937457]
 [ 0.          0.          0.          0.          0.        ]
 [ 0.          0.95827857  0.          0.87376511  0.        ]
 [ 0.98344586  0.          0.          0.96333499  0.75284338]
 [ 0.          0.          0.91466554  0.          0.8520395 ]]

The CSR representation can be very efficient for computations, but it is not as good for adding elements. For that, the LIL (List-In-List) representation is better:

# Create an empty LIL matrix and add some items
X_lil = sparse.lil_matrix((5, 5))

for i, j in np.random.randint(0, 5, (15, 2)):
    X_lil[i, j] = i + j

print(X_lil)

  (0, 1)	1.0
  (0, 3)	3.0
  (0, 4)	4.0
  (1, 0)	1.0
  (1, 1)	2.0
  (1, 2)	3.0
  (1, 3)	4.0
  (2, 1)	3.0
  (2, 2)	4.0
  (2, 4)	6.0
  (3, 0)	3.0
  (3, 2)	5.0
  (4, 3)	7.0

print(X_lil.toarray())

[[ 0.  1.  0.  3.  4.]
 [ 1.  2.  3.  4.  0.]
 [ 0.  3.  4.  0.  6.]
 [ 3.  0.  5.  0.  0.]
 [ 0.  0.  0.  7.  0.]]

Often, once an LIL matrix is created, it is useful to convert it to a CSR format (many scikit-learn algorithms require CSR or CSC format)

print(X_lil.tocsr())

  (0, 1)	1.0
  (0, 3)	3.0
  (0, 4)	4.0
  (1, 0)	1.0
  (1, 1)	2.0
  (1, 2)	3.0
  (1, 3)	4.0
  (2, 1)	3.0
  (2, 2)	4.0
  (2, 4)	6.0
  (3, 0)	3.0
  (3, 2)	5.0
  (4, 3)	7.0

The available sparse formats that can be useful for various problems:

CSR (compressed sparse row)
CSC (compressed sparse column)
BSR (block sparse row)
COO (coordinate)
DIA (diagonal)
DOK (dictionary of keys)
LIL (list in list)

The scipy.sparse submodule also has a lot of functions for sparse matrices including linear algebra, sparse solvers, graph algorithms, and much more.

Matplotlib¶

Another important part of machine learning is visualization of data. The most common tool for this in Python is matplotlib. It is an extremely flexible package, but we will go over some basics here.

First, something special to IPython notebook. We can turn on the "IPython inline" mode, which will make plots show up inline in the notebook.

%matplotlib inline

import matplotlib.pyplot as plt

# plotting a line
x = np.linspace(0, 10, 100)
plt.plot(x, np.sin(x))

[<matplotlib.lines.Line2D at 0x81a4c88>]

# scatter-plot points
x = np.random.normal(size=500)
y = np.random.normal(size=500)
plt.scatter(x, y)

<matplotlib.collections.PathCollection at 0x88d2da0>

# showing images
x = np.linspace(1, 12, 100)
y = x[:, np.newaxis]

im = y * np.sin(x) * np.cos(y)
print(im.shape)

(100, 100)

# imshow - note that origin is at the top-left by default!
plt.imshow(im)

<matplotlib.image.AxesImage at 0x8940160>

# Contour plot - note that origin here is at the bottom-left by default!
plt.contour(im)

<matplotlib.contour.QuadContourSet at 0x899cdd8>

# 3D plotting
from mpl_toolkits.mplot3d import Axes3D
ax = plt.axes(projection='3d')
xgrid, ygrid = np.meshgrid(x, y.ravel())
ax.plot_surface(xgrid, ygrid, im, cmap=plt.cm.jet, cstride=2, rstride=2, linewidth=0)

<mpl_toolkits.mplot3d.art3d.Poly3DCollection at 0x8901780>

There are many, many more plot types available. One useful way to explore these is by looking at the matplotlib gallery: http://matplotlib.org/gallery.html

You can test these examples out easily in the notebook: simply copy the Source Code link on each page, and put it in a notebook using the %load magic. For example:

# %load http://matplotlib.org/mpl_examples/pylab_examples/ellipse_collection.py
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.collections import EllipseCollection

x = np.arange(10)
y = np.arange(15)
X, Y = np.meshgrid(x, y)

XY = np.hstack((X.ravel()[:,np.newaxis], Y.ravel()[:,np.newaxis]))

ww = X/10.0
hh = Y/15.0
aa = X*9

fig, ax = plt.subplots()

ec = EllipseCollection(ww, hh, aa, units='x', offsets=XY,
                       transOffset=ax.transData)
ec.set_array((X+Y).ravel())
ax.add_collection(ec)
ax.autoscale_view()
ax.set_xlabel('X')
ax.set_ylabel('y')
cbar = plt.colorbar(ec)
cbar.set_label('X+Y')
plt.show()